Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 926902 Details for
Bug 1130315
Rubygem-Staypuft: After deploying HA neutron, neutron-openvswitch-agent reported failed. pcs resource cleanup <ID> removes it from the failed list.
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
openvswitch-agent log from one controller.
openvswitch-agent.log (text/x-log), 298.61 KB, created by
Alexander Chuzhoy
on 2014-08-14 20:25:49 UTC
(
hide
)
Description:
openvswitch-agent log from one controller.
Filename:
MIME Type:
Creator:
Alexander Chuzhoy
Created:
2014-08-14 20:25:49 UTC
Size:
298.61 KB
patch
obsolete
>2014-08-14 18:53:39.216 28402 INFO neutron.common.config [-] Logging enabled! >2014-08-14 18:53:39.803 28402 INFO neutron.openstack.common.rpc.common [req-d6b01ad2-9abb-4d2d-96f3-285c703c5dbb None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 18:53:39.849 28402 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-d6b01ad2-9abb-4d2d-96f3-285c703c5dbb None] Mapping physical network physnet-external to bridge br-ex >2014-08-14 18:53:39.874 28402 INFO neutron.openstack.common.rpc.common [req-d6b01ad2-9abb-4d2d-96f3-285c703c5dbb None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 18:53:42.277 28402 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent initialized successfully, now running... >2014-08-14 18:53:42.282 28402 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with plugin! >2014-08-14 18:53:42.356 28402 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent tunnel out of sync with plugin! >2014-08-14 18:53:42.370 28402 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 18:55:22.984 32609 INFO neutron.common.config [-] Logging enabled! >2014-08-14 18:55:23.545 32609 INFO neutron.openstack.common.rpc.common [req-cebacad5-43af-4f53-aa41-9f961849400b None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 18:55:23.591 32609 CRITICAL neutron [req-cebacad5-43af-4f53-aa41-9f961849400b None] 404: (NOT_FOUND - no exchange 'q-agent-notifier-network-delete_fanout' in vhost '/', (50, 20), None) >2014-08-14 18:55:23.591 32609 TRACE neutron Traceback (most recent call last): >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/bin/neutron-openvswitch-agent", line 10, in <module> >2014-08-14 18:55:23.591 32609 TRACE neutron sys.exit(main()) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1469, in main >2014-08-14 18:55:23.591 32609 TRACE neutron agent = OVSNeutronAgent(**agent_config) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 205, in __init__ >2014-08-14 18:55:23.591 32609 TRACE neutron self.setup_rpc() >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 277, in setup_rpc >2014-08-14 18:55:23.591 32609 TRACE neutron consumers) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/rpc.py", line 49, in create_consumers >2014-08-14 18:55:23.591 32609 TRACE neutron connection.create_consumer(topic_name, dispatcher, fanout=True) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/amqp.py", line 163, in create_consumer >2014-08-14 18:55:23.591 32609 TRACE neutron self.connection.create_consumer(topic, proxy, fanout) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 761, in create_consumer >2014-08-14 18:55:23.591 32609 TRACE neutron self.declare_fanout_consumer(topic, proxy_cb) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 714, in declare_fanout_consumer >2014-08-14 18:55:23.591 32609 TRACE neutron self.declare_consumer(FanoutConsumer, topic, callback) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 634, in declare_consumer >2014-08-14 18:55:23.591 32609 TRACE neutron return self.ensure(_connect_error, _declare_consumer) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 18:55:23.591 32609 TRACE neutron return method(*args, **kwargs) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 630, in _declare_consumer >2014-08-14 18:55:23.591 32609 TRACE neutron six.next(self.consumer_num)) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 302, in __init__ >2014-08-14 18:55:23.591 32609 TRACE neutron **options) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 133, in __init__ >2014-08-14 18:55:23.591 32609 TRACE neutron self.reconnect(channel) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 140, in reconnect >2014-08-14 18:55:23.591 32609 TRACE neutron self.queue.declare() >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 497, in declare >2014-08-14 18:55:23.591 32609 TRACE neutron self.queue_bind(nowait) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 530, in queue_bind >2014-08-14 18:55:23.591 32609 TRACE neutron self.binding_arguments, nowait=nowait) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 540, in bind_to >2014-08-14 18:55:23.591 32609 TRACE neutron nowait=nowait) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 986, in queue_bind >2014-08-14 18:55:23.591 32609 TRACE neutron (50, 21), # Channel.queue_bind_ok >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 18:55:23.591 32609 TRACE neutron return self.dispatch_method(method_sig, args, content) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 18:55:23.591 32609 TRACE neutron return amqp_method(self, args) >2014-08-14 18:55:23.591 32609 TRACE neutron File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 18:55:23.591 32609 TRACE neutron raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 18:55:23.591 32609 TRACE neutron ChannelError: 404: (NOT_FOUND - no exchange 'q-agent-notifier-network-delete_fanout' in vhost '/', (50, 20), None) >2014-08-14 18:55:23.591 32609 TRACE neutron >2014-08-14 19:02:15.311 14067 INFO neutron.common.config [-] Logging enabled! >2014-08-14 19:02:15.899 14067 INFO neutron.openstack.common.rpc.common [req-4b65cf49-3645-4e2f-be0b-b107f7e887df None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:02:15.927 14067 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-4b65cf49-3645-4e2f-be0b-b107f7e887df None] Mapping physical network physnet-external to bridge br-ex >2014-08-14 19:02:15.950 14067 INFO neutron.openstack.common.rpc.common [req-4b65cf49-3645-4e2f-be0b-b107f7e887df None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:02:18.389 14067 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-4b65cf49-3645-4e2f-be0b-b107f7e887df None] Agent initialized successfully, now running... >2014-08-14 19:02:18.394 14067 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-4b65cf49-3645-4e2f-be0b-b107f7e887df None] Agent out of sync with plugin! >2014-08-14 19:02:18.467 14067 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-4b65cf49-3645-4e2f-be0b-b107f7e887df None] Agent tunnel out of sync with plugin! >2014-08-14 19:02:18.480 14067 INFO neutron.openstack.common.rpc.common [req-4b65cf49-3645-4e2f-be0b-b107f7e887df None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:04:18.974 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:04:18.974 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:04:18.977 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:04:18.986 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:04:18.986 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:04:18.987 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:04:19.015 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:04:19.081 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:06:19.028 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:06:19.028 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:06:19.029 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:06:19.050 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:06:19.100 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:06:19.100 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:06:19.101 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:06:19.166 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:08:19.064 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:08:19.064 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:08:19.065 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:08:19.086 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:08:19.183 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:08:19.183 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:08:19.184 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:08:19.232 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:08:19.234 14067 ERROR root [-] Unexpected exception occurred 1 time(s)... retrying. >2014-08-14 19:08:19.234 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:08:19.234 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:08:19.234 14067 TRACE root self.consume() >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:08:19.234 14067 TRACE root six.next(it) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:08:19.234 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:08:19.234 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:08:19.234 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:08:19.234 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:08:19.234 14067 TRACE root nowait=nowait) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:08:19.234 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:08:19.234 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:08:19.234 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:08:19.234 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:08:19.234 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:08:19.234 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:08:19.234 14067 TRACE root >2014-08-14 19:09:20.359 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:09:20.359 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:09:20.359 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:09:20.359 14067 TRACE root self.consume() >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:09:20.359 14067 TRACE root six.next(it) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:09:20.359 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:09:20.359 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:09:20.359 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:09:20.359 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:09:20.359 14067 TRACE root nowait=nowait) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:09:20.359 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:09:20.359 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:09:20.359 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:09:20.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:09:20.359 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:09:20.359 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:09:20.359 14067 TRACE root >2014-08-14 19:10:19.098 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:10:19.098 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:10:19.099 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:10:19.119 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:10:21.485 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:10:21.485 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:10:21.485 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:10:21.485 14067 TRACE root self.consume() >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:10:21.485 14067 TRACE root six.next(it) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:10:21.485 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:10:21.485 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:10:21.485 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:10:21.485 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:10:21.485 14067 TRACE root nowait=nowait) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:10:21.485 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:10:21.485 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:10:21.485 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:10:21.485 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:10:21.485 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:10:21.485 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:10:21.485 14067 TRACE root >2014-08-14 19:11:22.615 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:11:22.615 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:11:22.615 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:11:22.615 14067 TRACE root self.consume() >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:11:22.615 14067 TRACE root six.next(it) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:11:22.615 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:11:22.615 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:11:22.615 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:11:22.615 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:11:22.615 14067 TRACE root nowait=nowait) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:11:22.615 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:11:22.615 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:11:22.615 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:11:22.615 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:11:22.615 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:11:22.615 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:11:22.615 14067 TRACE root >2014-08-14 19:12:19.137 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:12:19.137 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:12:19.138 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:12:19.159 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:12:23.756 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:12:23.756 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:12:23.756 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:12:23.756 14067 TRACE root self.consume() >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:12:23.756 14067 TRACE root six.next(it) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:12:23.756 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:12:23.756 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:12:23.756 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:12:23.756 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:12:23.756 14067 TRACE root nowait=nowait) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:12:23.756 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:12:23.756 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:12:23.756 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:12:23.756 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:12:23.756 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:12:23.756 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:12:23.756 14067 TRACE root >2014-08-14 19:13:24.877 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:13:24.877 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:13:24.877 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:13:24.877 14067 TRACE root self.consume() >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:13:24.877 14067 TRACE root six.next(it) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:13:24.877 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:13:24.877 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:13:24.877 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:13:24.877 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:13:24.877 14067 TRACE root nowait=nowait) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:13:24.877 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:13:24.877 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:13:24.877 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:13:24.877 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:13:24.877 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:13:24.877 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:13:24.877 14067 TRACE root >2014-08-14 19:14:19.176 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:14:19.176 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:14:19.177 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:14:19.197 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:14:25.996 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:14:25.996 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:14:25.996 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:14:25.996 14067 TRACE root self.consume() >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:14:25.996 14067 TRACE root six.next(it) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:14:25.996 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:14:25.996 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:14:25.996 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:14:25.996 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:14:25.996 14067 TRACE root nowait=nowait) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:14:25.996 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:14:25.996 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:14:25.996 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:14:25.996 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:14:25.996 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:14:25.996 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:14:25.996 14067 TRACE root >2014-08-14 19:15:26.113 14067 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying. >2014-08-14 19:15:26.113 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:15:26.113 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:15:26.113 14067 TRACE root self.consume() >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:15:26.113 14067 TRACE root six.next(it) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:15:26.113 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:15:26.113 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:15:26.113 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:15:26.113 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:15:26.113 14067 TRACE root nowait=nowait) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:15:26.113 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:15:26.113 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:15:26.113 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:15:26.113 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:15:26.113 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:15:26.113 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:15:26.113 14067 TRACE root >2014-08-14 19:16:19.209 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:16:19.209 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:16:19.210 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:16:19.230 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:16:27.237 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:16:27.237 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:16:27.237 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:16:27.237 14067 TRACE root self.consume() >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:16:27.237 14067 TRACE root six.next(it) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:16:27.237 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:16:27.237 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:16:27.237 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:16:27.237 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:16:27.237 14067 TRACE root nowait=nowait) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:16:27.237 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:16:27.237 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:16:27.237 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:16:27.237 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:16:27.237 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:16:27.237 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:16:27.237 14067 TRACE root >2014-08-14 19:17:28.359 14067 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:17:28.359 14067 TRACE root Traceback (most recent call last): >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:17:28.359 14067 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:17:28.359 14067 TRACE root self.consume() >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:17:28.359 14067 TRACE root six.next(it) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:17:28.359 14067 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:17:28.359 14067 TRACE root return method(*args, **kwargs) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:17:28.359 14067 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:17:28.359 14067 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:17:28.359 14067 TRACE root nowait=nowait) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:17:28.359 14067 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:17:28.359 14067 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:17:28.359 14067 TRACE root return amqp_method(self, args) >2014-08-14 19:17:28.359 14067 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:17:28.359 14067 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:17:28.359 14067 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_177d572ab6d94d6a81049213519e15ec' in vhost '/', (60, 20), None) >2014-08-14 19:17:28.359 14067 TRACE root >2014-08-14 19:18:19.250 14067 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:18:19.250 14067 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:18:19.251 14067 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:18:19.272 14067 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:32:15.781 30119 INFO neutron.common.config [-] Logging enabled! >2014-08-14 19:32:16.359 30119 INFO neutron.openstack.common.rpc.common [req-1fe7bcf4-7664-45ff-8e18-e6ff6233556d None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:32:16.395 30119 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-1fe7bcf4-7664-45ff-8e18-e6ff6233556d None] Mapping physical network physnet-external to bridge br-ex >2014-08-14 19:32:16.421 30119 INFO neutron.openstack.common.rpc.common [req-1fe7bcf4-7664-45ff-8e18-e6ff6233556d None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:32:18.875 30119 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-1fe7bcf4-7664-45ff-8e18-e6ff6233556d None] Agent initialized successfully, now running... >2014-08-14 19:32:18.881 30119 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-1fe7bcf4-7664-45ff-8e18-e6ff6233556d None] Agent out of sync with plugin! >2014-08-14 19:32:18.955 30119 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-1fe7bcf4-7664-45ff-8e18-e6ff6233556d None] Agent tunnel out of sync with plugin! >2014-08-14 19:32:18.969 30119 INFO neutron.openstack.common.rpc.common [req-1fe7bcf4-7664-45ff-8e18-e6ff6233556d None] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:34:19.916 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:34:19.916 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:34:19.919 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:34:19.928 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:34:19.928 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:34:19.929 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:34:19.954 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:34:20.074 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:34:20.075 30119 ERROR root [-] Unexpected exception occurred 1 time(s)... retrying. >2014-08-14 19:34:20.075 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:34:20.075 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:34:20.075 30119 TRACE root self.consume() >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:34:20.075 30119 TRACE root six.next(it) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:34:20.075 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:34:20.075 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:34:20.075 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:34:20.075 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:34:20.075 30119 TRACE root nowait=nowait) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:34:20.075 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:34:20.075 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:34:20.075 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:34:20.075 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:34:20.075 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:34:20.075 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:34:20.075 30119 TRACE root >2014-08-14 19:35:21.221 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:35:21.221 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:35:21.221 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:35:21.221 30119 TRACE root self.consume() >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:35:21.221 30119 TRACE root six.next(it) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:35:21.221 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:35:21.221 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:35:21.221 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:35:21.221 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:35:21.221 30119 TRACE root nowait=nowait) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:35:21.221 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:35:21.221 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:35:21.221 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:35:21.221 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:35:21.221 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:35:21.221 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:35:21.221 30119 TRACE root >2014-08-14 19:36:19.959 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:36:19.959 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:36:19.959 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:36:19.983 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:36:22.357 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:36:22.357 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:36:22.357 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:36:22.357 30119 TRACE root self.consume() >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:36:22.357 30119 TRACE root six.next(it) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:36:22.357 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:36:22.357 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:36:22.357 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:36:22.357 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:36:22.357 30119 TRACE root nowait=nowait) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:36:22.357 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:36:22.357 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:36:22.357 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:36:22.357 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:36:22.357 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:36:22.357 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:36:22.357 30119 TRACE root >2014-08-14 19:37:23.485 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:37:23.485 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:37:23.485 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:37:23.485 30119 TRACE root self.consume() >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:37:23.485 30119 TRACE root six.next(it) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:37:23.485 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:37:23.485 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:37:23.485 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:37:23.485 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:37:23.485 30119 TRACE root nowait=nowait) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:37:23.485 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:37:23.485 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:37:23.485 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:37:23.485 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:37:23.485 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:37:23.485 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:37:23.485 30119 TRACE root >2014-08-14 19:38:20.000 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:38:20.000 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:38:20.001 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:38:20.021 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:38:24.618 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:38:24.618 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:38:24.618 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:38:24.618 30119 TRACE root self.consume() >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:38:24.618 30119 TRACE root six.next(it) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:38:24.618 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:38:24.618 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:38:24.618 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:38:24.618 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:38:24.618 30119 TRACE root nowait=nowait) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:38:24.618 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:38:24.618 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:38:24.618 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:38:24.618 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:38:24.618 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:38:24.618 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:38:24.618 30119 TRACE root >2014-08-14 19:39:25.752 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:39:25.752 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:39:25.752 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:39:25.752 30119 TRACE root self.consume() >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:39:25.752 30119 TRACE root six.next(it) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:39:25.752 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:39:25.752 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:39:25.752 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:39:25.752 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:39:25.752 30119 TRACE root nowait=nowait) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:39:25.752 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:39:25.752 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:39:25.752 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:39:25.752 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:39:25.752 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:39:25.752 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:39:25.752 30119 TRACE root >2014-08-14 19:40:20.035 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:40:20.035 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:40:20.036 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:40:20.055 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:40:26.887 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:40:26.887 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:40:26.887 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:40:26.887 30119 TRACE root self.consume() >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:40:26.887 30119 TRACE root six.next(it) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:40:26.887 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:40:26.887 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:40:26.887 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:40:26.887 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:40:26.887 30119 TRACE root nowait=nowait) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:40:26.887 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:40:26.887 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:40:26.887 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:40:26.887 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:40:26.887 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:40:26.887 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:40:26.887 30119 TRACE root >2014-08-14 19:41:27.026 30119 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying. >2014-08-14 19:41:27.026 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:41:27.026 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:41:27.026 30119 TRACE root self.consume() >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:41:27.026 30119 TRACE root six.next(it) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:41:27.026 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:41:27.026 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:41:27.026 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:41:27.026 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:41:27.026 30119 TRACE root nowait=nowait) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:41:27.026 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:41:27.026 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:41:27.026 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:41:27.026 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:41:27.026 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:41:27.026 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:41:27.026 30119 TRACE root >2014-08-14 19:42:20.077 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:42:20.077 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:42:20.078 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:42:20.097 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:42:28.170 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:42:28.170 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:42:28.170 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:42:28.170 30119 TRACE root self.consume() >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:42:28.170 30119 TRACE root six.next(it) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:42:28.170 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:42:28.170 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:42:28.170 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:42:28.170 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:42:28.170 30119 TRACE root nowait=nowait) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:42:28.170 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:42:28.170 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:42:28.170 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:42:28.170 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:42:28.170 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:42:28.170 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:42:28.170 30119 TRACE root >2014-08-14 19:43:29.318 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:43:29.318 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:43:29.318 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:43:29.318 30119 TRACE root self.consume() >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:43:29.318 30119 TRACE root six.next(it) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:43:29.318 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:43:29.318 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:43:29.318 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:43:29.318 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:43:29.318 30119 TRACE root nowait=nowait) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:43:29.318 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:43:29.318 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:43:29.318 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:43:29.318 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:43:29.318 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:43:29.318 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:43:29.318 30119 TRACE root >2014-08-14 19:44:20.108 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:44:20.108 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:44:20.109 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:44:20.129 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:44:30.450 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:44:30.450 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:44:30.450 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:44:30.450 30119 TRACE root self.consume() >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:44:30.450 30119 TRACE root six.next(it) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:44:30.450 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:44:30.450 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:44:30.450 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:44:30.450 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:44:30.450 30119 TRACE root nowait=nowait) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:44:30.450 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:44:30.450 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:44:30.450 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:44:30.450 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:44:30.450 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:44:30.450 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:44:30.450 30119 TRACE root >2014-08-14 19:45:31.579 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:45:31.579 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:45:31.579 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:45:31.579 30119 TRACE root self.consume() >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:45:31.579 30119 TRACE root six.next(it) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:45:31.579 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:45:31.579 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:45:31.579 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:45:31.579 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:45:31.579 30119 TRACE root nowait=nowait) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:45:31.579 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:45:31.579 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:45:31.579 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:45:31.579 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:45:31.579 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:45:31.579 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:45:31.579 30119 TRACE root >2014-08-14 19:46:20.145 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:46:20.145 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:46:20.146 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:46:20.166 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:46:32.709 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:46:32.709 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:46:32.709 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:46:32.709 30119 TRACE root self.consume() >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:46:32.709 30119 TRACE root six.next(it) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:46:32.709 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:46:32.709 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:46:32.709 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:46:32.709 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:46:32.709 30119 TRACE root nowait=nowait) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:46:32.709 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:46:32.709 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:46:32.709 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:46:32.709 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:46:32.709 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:46:32.709 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:46:32.709 30119 TRACE root >2014-08-14 19:47:33.843 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:47:33.843 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:47:33.843 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:47:33.843 30119 TRACE root self.consume() >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:47:33.843 30119 TRACE root six.next(it) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:47:33.843 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:47:33.843 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:47:33.843 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:47:33.843 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:47:33.843 30119 TRACE root nowait=nowait) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:47:33.843 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:47:33.843 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:47:33.843 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:47:33.843 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:47:33.843 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:47:33.843 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:47:33.843 30119 TRACE root >2014-08-14 19:48:20.185 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:48:20.185 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:48:20.186 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:48:20.206 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:48:34.980 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:48:34.980 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:48:34.980 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:48:34.980 30119 TRACE root self.consume() >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:48:34.980 30119 TRACE root six.next(it) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:48:34.980 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:48:34.980 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:48:34.980 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:48:34.980 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:48:34.980 30119 TRACE root nowait=nowait) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:48:34.980 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:48:34.980 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:48:34.980 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:48:34.980 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:48:34.980 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:48:34.980 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:48:34.980 30119 TRACE root >2014-08-14 19:49:35.113 30119 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying. >2014-08-14 19:49:35.113 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:49:35.113 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:49:35.113 30119 TRACE root self.consume() >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:49:35.113 30119 TRACE root six.next(it) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:49:35.113 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:49:35.113 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:49:35.113 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:49:35.113 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:49:35.113 30119 TRACE root nowait=nowait) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:49:35.113 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:49:35.113 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:49:35.113 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:49:35.113 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:49:35.113 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:49:35.113 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:49:35.113 30119 TRACE root >2014-08-14 19:50:20.219 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:50:20.219 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:50:20.220 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:50:20.240 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:50:36.255 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:50:36.255 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:50:36.255 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:50:36.255 30119 TRACE root self.consume() >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:50:36.255 30119 TRACE root six.next(it) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:50:36.255 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:50:36.255 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:50:36.255 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:50:36.255 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:50:36.255 30119 TRACE root nowait=nowait) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:50:36.255 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:50:36.255 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:50:36.255 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:50:36.255 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:50:36.255 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:50:36.255 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:50:36.255 30119 TRACE root >2014-08-14 19:51:37.405 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:51:37.405 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:51:37.405 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:51:37.405 30119 TRACE root self.consume() >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:51:37.405 30119 TRACE root six.next(it) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:51:37.405 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:51:37.405 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:51:37.405 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:51:37.405 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:51:37.405 30119 TRACE root nowait=nowait) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:51:37.405 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:51:37.405 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:51:37.405 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:51:37.405 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:51:37.405 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:51:37.405 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:51:37.405 30119 TRACE root >2014-08-14 19:52:20.252 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:52:20.252 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:52:20.253 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:52:20.273 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:52:38.536 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:52:38.536 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:52:38.536 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:52:38.536 30119 TRACE root self.consume() >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:52:38.536 30119 TRACE root six.next(it) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:52:38.536 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:52:38.536 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:52:38.536 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:52:38.536 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:52:38.536 30119 TRACE root nowait=nowait) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:52:38.536 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:52:38.536 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:52:38.536 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:52:38.536 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:52:38.536 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:52:38.536 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:52:38.536 30119 TRACE root >2014-08-14 19:53:39.668 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:53:39.668 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:53:39.668 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:53:39.668 30119 TRACE root self.consume() >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:53:39.668 30119 TRACE root six.next(it) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:53:39.668 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:53:39.668 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:53:39.668 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:53:39.668 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:53:39.668 30119 TRACE root nowait=nowait) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:53:39.668 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:53:39.668 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:53:39.668 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:53:39.668 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:53:39.668 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:53:39.668 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:53:39.668 30119 TRACE root >2014-08-14 19:54:20.292 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:54:20.292 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:54:20.293 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:54:20.312 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:54:40.801 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:54:40.801 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:54:40.801 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:54:40.801 30119 TRACE root self.consume() >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:54:40.801 30119 TRACE root six.next(it) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:54:40.801 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:54:40.801 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:54:40.801 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:54:40.801 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:54:40.801 30119 TRACE root nowait=nowait) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:54:40.801 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:54:40.801 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:54:40.801 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:54:40.801 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:54:40.801 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:54:40.801 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:54:40.801 30119 TRACE root >2014-08-14 19:55:41.937 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:55:41.937 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:55:41.937 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:55:41.937 30119 TRACE root self.consume() >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:55:41.937 30119 TRACE root six.next(it) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:55:41.937 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:55:41.937 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:55:41.937 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:55:41.937 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:55:41.937 30119 TRACE root nowait=nowait) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:55:41.937 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:55:41.937 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:55:41.937 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:55:41.937 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:55:41.937 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:55:41.937 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:55:41.937 30119 TRACE root >2014-08-14 19:56:20.332 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:56:20.332 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:56:20.333 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:56:20.353 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:56:42.072 30119 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying. >2014-08-14 19:56:42.072 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:56:42.072 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:56:42.072 30119 TRACE root self.consume() >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:56:42.072 30119 TRACE root six.next(it) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:56:42.072 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:56:42.072 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:56:42.072 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:56:42.072 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:56:42.072 30119 TRACE root nowait=nowait) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:56:42.072 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:56:42.072 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:56:42.072 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:56:42.072 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:56:42.072 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:56:42.072 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:56:42.072 30119 TRACE root >2014-08-14 19:57:43.216 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:57:43.216 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:57:43.216 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:57:43.216 30119 TRACE root self.consume() >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:57:43.216 30119 TRACE root six.next(it) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:57:43.216 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:57:43.216 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:57:43.216 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:57:43.216 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:57:43.216 30119 TRACE root nowait=nowait) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:57:43.216 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:57:43.216 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:57:43.216 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:57:43.216 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:57:43.216 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:57:43.216 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:57:43.216 30119 TRACE root >2014-08-14 19:58:20.367 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 19:58:20.367 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 19:58:20.368 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 19:58:20.387 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 19:58:44.356 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:58:44.356 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:58:44.356 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:58:44.356 30119 TRACE root self.consume() >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:58:44.356 30119 TRACE root six.next(it) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:58:44.356 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:58:44.356 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:58:44.356 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:58:44.356 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:58:44.356 30119 TRACE root nowait=nowait) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:58:44.356 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:58:44.356 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:58:44.356 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:58:44.356 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:58:44.356 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:58:44.356 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:58:44.356 30119 TRACE root >2014-08-14 19:59:45.497 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 19:59:45.497 30119 TRACE root Traceback (most recent call last): >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 19:59:45.497 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 19:59:45.497 30119 TRACE root self.consume() >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 19:59:45.497 30119 TRACE root six.next(it) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 19:59:45.497 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 19:59:45.497 30119 TRACE root return method(*args, **kwargs) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 19:59:45.497 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 19:59:45.497 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 19:59:45.497 30119 TRACE root nowait=nowait) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 19:59:45.497 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 19:59:45.497 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 19:59:45.497 30119 TRACE root return amqp_method(self, args) >2014-08-14 19:59:45.497 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 19:59:45.497 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 19:59:45.497 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 19:59:45.497 30119 TRACE root >2014-08-14 20:00:20.402 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:00:20.402 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:00:20.403 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:00:20.422 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:00:46.630 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:00:46.630 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:00:46.630 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:00:46.630 30119 TRACE root self.consume() >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:00:46.630 30119 TRACE root six.next(it) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:00:46.630 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:00:46.630 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:00:46.630 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:00:46.630 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:00:46.630 30119 TRACE root nowait=nowait) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:00:46.630 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:00:46.630 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:00:46.630 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:00:46.630 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:00:46.630 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:00:46.630 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:00:46.630 30119 TRACE root >2014-08-14 20:01:47.761 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:01:47.761 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:01:47.761 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:01:47.761 30119 TRACE root self.consume() >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:01:47.761 30119 TRACE root six.next(it) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:01:47.761 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:01:47.761 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:01:47.761 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:01:47.761 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:01:47.761 30119 TRACE root nowait=nowait) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:01:47.761 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:01:47.761 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:01:47.761 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:01:47.761 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:01:47.761 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:01:47.761 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:01:47.761 30119 TRACE root >2014-08-14 20:02:20.439 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:02:20.439 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:02:20.440 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:02:20.459 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:02:48.890 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:02:48.890 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:02:48.890 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:02:48.890 30119 TRACE root self.consume() >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:02:48.890 30119 TRACE root six.next(it) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:02:48.890 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:02:48.890 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:02:48.890 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:02:48.890 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:02:48.890 30119 TRACE root nowait=nowait) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:02:48.890 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:02:48.890 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:02:48.890 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:02:48.890 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:02:48.890 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:02:48.890 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:02:48.890 30119 TRACE root >2014-08-14 20:03:49.022 30119 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying. >2014-08-14 20:03:49.022 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:03:49.022 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:03:49.022 30119 TRACE root self.consume() >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:03:49.022 30119 TRACE root six.next(it) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:03:49.022 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:03:49.022 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:03:49.022 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:03:49.022 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:03:49.022 30119 TRACE root nowait=nowait) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:03:49.022 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:03:49.022 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:03:49.022 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:03:49.022 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:03:49.022 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:03:49.022 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:03:49.022 30119 TRACE root >2014-08-14 20:04:20.474 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:04:20.474 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:04:20.475 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:04:20.496 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:04:50.159 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:04:50.159 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:04:50.159 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:04:50.159 30119 TRACE root self.consume() >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:04:50.159 30119 TRACE root six.next(it) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:04:50.159 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:04:50.159 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:04:50.159 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:04:50.159 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:04:50.159 30119 TRACE root nowait=nowait) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:04:50.159 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:04:50.159 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:04:50.159 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:04:50.159 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:04:50.159 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:04:50.159 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:04:50.159 30119 TRACE root >2014-08-14 20:05:51.299 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:05:51.299 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:05:51.299 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:05:51.299 30119 TRACE root self.consume() >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:05:51.299 30119 TRACE root six.next(it) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:05:51.299 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:05:51.299 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:05:51.299 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:05:51.299 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:05:51.299 30119 TRACE root nowait=nowait) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:05:51.299 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:05:51.299 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:05:51.299 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:05:51.299 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:05:51.299 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:05:51.299 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:05:51.299 30119 TRACE root >2014-08-14 20:06:20.509 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:06:20.509 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:06:20.510 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:06:20.530 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:06:52.447 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:06:52.447 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:06:52.447 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:06:52.447 30119 TRACE root self.consume() >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:06:52.447 30119 TRACE root six.next(it) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:06:52.447 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:06:52.447 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:06:52.447 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:06:52.447 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:06:52.447 30119 TRACE root nowait=nowait) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:06:52.447 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:06:52.447 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:06:52.447 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:06:52.447 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:06:52.447 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:06:52.447 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:06:52.447 30119 TRACE root >2014-08-14 20:07:53.583 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:07:53.583 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:07:53.583 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:07:53.583 30119 TRACE root self.consume() >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:07:53.583 30119 TRACE root six.next(it) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:07:53.583 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:07:53.583 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:07:53.583 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:07:53.583 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:07:53.583 30119 TRACE root nowait=nowait) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:07:53.583 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:07:53.583 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:07:53.583 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:07:53.583 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:07:53.583 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:07:53.583 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:07:53.583 30119 TRACE root >2014-08-14 20:08:20.541 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:08:20.541 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:08:20.542 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:08:20.562 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:08:54.716 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:08:54.716 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:08:54.716 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:08:54.716 30119 TRACE root self.consume() >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:08:54.716 30119 TRACE root six.next(it) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:08:54.716 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:08:54.716 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:08:54.716 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:08:54.716 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:08:54.716 30119 TRACE root nowait=nowait) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:08:54.716 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:08:54.716 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:08:54.716 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:08:54.716 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:08:54.716 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:08:54.716 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:08:54.716 30119 TRACE root >2014-08-14 20:09:55.851 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:09:55.851 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:09:55.851 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:09:55.851 30119 TRACE root self.consume() >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:09:55.851 30119 TRACE root six.next(it) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:09:55.851 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:09:55.851 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:09:55.851 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:09:55.851 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:09:55.851 30119 TRACE root nowait=nowait) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:09:55.851 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:09:55.851 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:09:55.851 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:09:55.851 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:09:55.851 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:09:55.851 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:09:55.851 30119 TRACE root >2014-08-14 20:10:20.582 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:10:20.582 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:10:20.583 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:10:20.603 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:10:56.983 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:10:56.983 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:10:56.983 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:10:56.983 30119 TRACE root self.consume() >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:10:56.983 30119 TRACE root six.next(it) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:10:56.983 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:10:56.983 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:10:56.983 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:10:56.983 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:10:56.983 30119 TRACE root nowait=nowait) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:10:56.983 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:10:56.983 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:10:56.983 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:10:56.983 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:10:56.983 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:10:56.983 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:10:56.983 30119 TRACE root >2014-08-14 20:11:57.117 30119 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying. >2014-08-14 20:11:57.117 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:11:57.117 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:11:57.117 30119 TRACE root self.consume() >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:11:57.117 30119 TRACE root six.next(it) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:11:57.117 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:11:57.117 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:11:57.117 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:11:57.117 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:11:57.117 30119 TRACE root nowait=nowait) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:11:57.117 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:11:57.117 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:11:57.117 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:11:57.117 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:11:57.117 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:11:57.117 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:11:57.117 30119 TRACE root >2014-08-14 20:12:20.615 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:12:20.615 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:12:20.616 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:12:20.636 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:12:58.252 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:12:58.252 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:12:58.252 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:12:58.252 30119 TRACE root self.consume() >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:12:58.252 30119 TRACE root six.next(it) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:12:58.252 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:12:58.252 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:12:58.252 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:12:58.252 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:12:58.252 30119 TRACE root nowait=nowait) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:12:58.252 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:12:58.252 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:12:58.252 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:12:58.252 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:12:58.252 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:12:58.252 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:12:58.252 30119 TRACE root >2014-08-14 20:13:59.389 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:13:59.389 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:13:59.389 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:13:59.389 30119 TRACE root self.consume() >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:13:59.389 30119 TRACE root six.next(it) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:13:59.389 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:13:59.389 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:13:59.389 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:13:59.389 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:13:59.389 30119 TRACE root nowait=nowait) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:13:59.389 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:13:59.389 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:13:59.389 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:13:59.389 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:13:59.389 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:13:59.389 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:13:59.389 30119 TRACE root >2014-08-14 20:14:20.650 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:14:20.650 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:14:20.651 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:14:20.671 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:15:00.531 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:15:00.531 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:15:00.531 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:15:00.531 30119 TRACE root self.consume() >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:15:00.531 30119 TRACE root six.next(it) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:15:00.531 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:15:00.531 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:15:00.531 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:15:00.531 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:15:00.531 30119 TRACE root nowait=nowait) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:15:00.531 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:15:00.531 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:15:00.531 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:15:00.531 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:15:00.531 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:15:00.531 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:15:00.531 30119 TRACE root >2014-08-14 20:16:01.666 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:16:01.666 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:16:01.666 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:16:01.666 30119 TRACE root self.consume() >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:16:01.666 30119 TRACE root six.next(it) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:16:01.666 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:16:01.666 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:16:01.666 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:16:01.666 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:16:01.666 30119 TRACE root nowait=nowait) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:16:01.666 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:16:01.666 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:16:01.666 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:16:01.666 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:16:01.666 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:16:01.666 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:16:01.666 30119 TRACE root >2014-08-14 20:16:20.692 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:16:20.692 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:16:20.693 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:16:20.713 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:17:02.796 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:17:02.796 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:17:02.796 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:17:02.796 30119 TRACE root self.consume() >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:17:02.796 30119 TRACE root six.next(it) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:17:02.796 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:17:02.796 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:17:02.796 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:17:02.796 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:17:02.796 30119 TRACE root nowait=nowait) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:17:02.796 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:17:02.796 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:17:02.796 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:17:02.796 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:17:02.796 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:17:02.796 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:17:02.796 30119 TRACE root >2014-08-14 20:18:03.926 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:18:03.926 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:18:03.926 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:18:03.926 30119 TRACE root self.consume() >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:18:03.926 30119 TRACE root six.next(it) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:18:03.926 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:18:03.926 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:18:03.926 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:18:03.926 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:18:03.926 30119 TRACE root nowait=nowait) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:18:03.926 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:18:03.926 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:18:03.926 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:18:03.926 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:18:03.926 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:18:03.926 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:18:03.926 30119 TRACE root >2014-08-14 20:18:20.727 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:18:20.727 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:18:20.728 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:18:20.749 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:19:04.058 30119 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying. >2014-08-14 20:19:04.058 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:19:04.058 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:19:04.058 30119 TRACE root self.consume() >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:19:04.058 30119 TRACE root six.next(it) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:19:04.058 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:19:04.058 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:19:04.058 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:19:04.058 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:19:04.058 30119 TRACE root nowait=nowait) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:19:04.058 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:19:04.058 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:19:04.058 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:19:04.058 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:19:04.058 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:19:04.058 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:19:04.058 30119 TRACE root >2014-08-14 20:20:05.194 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:20:05.194 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:20:05.194 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:20:05.194 30119 TRACE root self.consume() >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:20:05.194 30119 TRACE root six.next(it) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:20:05.194 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:20:05.194 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:20:05.194 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:20:05.194 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:20:05.194 30119 TRACE root nowait=nowait) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:20:05.194 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:20:05.194 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:20:05.194 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:20:05.194 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:20:05.194 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:20:05.194 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:20:05.194 30119 TRACE root >2014-08-14 20:20:20.766 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:20:20.766 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:20:20.767 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:20:20.788 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:21:06.334 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:21:06.334 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:21:06.334 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:21:06.334 30119 TRACE root self.consume() >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:21:06.334 30119 TRACE root six.next(it) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:21:06.334 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:21:06.334 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:21:06.334 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:21:06.334 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:21:06.334 30119 TRACE root nowait=nowait) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:21:06.334 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:21:06.334 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:21:06.334 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:21:06.334 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:21:06.334 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:21:06.334 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:21:06.334 30119 TRACE root >2014-08-14 20:22:07.468 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:22:07.468 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:22:07.468 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:22:07.468 30119 TRACE root self.consume() >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:22:07.468 30119 TRACE root six.next(it) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:22:07.468 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:22:07.468 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:22:07.468 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:22:07.468 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:22:07.468 30119 TRACE root nowait=nowait) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:22:07.468 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:22:07.468 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:22:07.468 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:22:07.468 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:22:07.468 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:22:07.468 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:22:07.468 30119 TRACE root >2014-08-14 20:22:20.808 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:22:20.808 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:22:20.809 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:22:20.829 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:23:08.617 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:23:08.617 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:23:08.617 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:23:08.617 30119 TRACE root self.consume() >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:23:08.617 30119 TRACE root six.next(it) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:23:08.617 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:23:08.617 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:23:08.617 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:23:08.617 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:23:08.617 30119 TRACE root nowait=nowait) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:23:08.617 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:23:08.617 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:23:08.617 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:23:08.617 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:23:08.617 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:23:08.617 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:23:08.617 30119 TRACE root >2014-08-14 20:24:09.750 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:24:09.750 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:24:09.750 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:24:09.750 30119 TRACE root self.consume() >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:24:09.750 30119 TRACE root six.next(it) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:24:09.750 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:24:09.750 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:24:09.750 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:24:09.750 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:24:09.750 30119 TRACE root nowait=nowait) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:24:09.750 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:24:09.750 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:24:09.750 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:24:09.750 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:24:09.750 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:24:09.750 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:24:09.750 30119 TRACE root >2014-08-14 20:24:20.846 30119 ERROR neutron.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last): >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common return method(*args, **kwargs) >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 659, in _consume >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 281, in drain_events >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common return connection.drain_events(**kwargs) >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common chanmap, None, timeout=timeout, >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout) >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 292, in read_timeout >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common return self.method_reader.read_method() >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 192, in read_method >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common raise m >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common IOError: Socket closed >2014-08-14 20:24:20.846 30119 TRACE neutron.openstack.common.rpc.common >2014-08-14 20:24:20.847 30119 INFO neutron.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.0.99:5672 >2014-08-14 20:24:20.867 30119 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.0.99:5672 >2014-08-14 20:25:10.882 30119 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying. >2014-08-14 20:25:10.882 30119 TRACE root Traceback (most recent call last): >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func >2014-08-14 20:25:10.882 30119 TRACE root return infunc(*args, **kwargs) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread >2014-08-14 20:25:10.882 30119 TRACE root self.consume() >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume >2014-08-14 20:25:10.882 30119 TRACE root six.next(it) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume >2014-08-14 20:25:10.882 30119 TRACE root yield self.ensure(_error_callback, _consume) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure >2014-08-14 20:25:10.882 30119 TRACE root return method(*args, **kwargs) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume >2014-08-14 20:25:10.882 30119 TRACE root queues_tail.consume(nowait=False) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume >2014-08-14 20:25:10.882 30119 TRACE root self.queue.consume(*args, callback=_callback, **options) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume >2014-08-14 20:25:10.882 30119 TRACE root nowait=nowait) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume >2014-08-14 20:25:10.882 30119 TRACE root (60, 21), # Channel.basic_consume_ok >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait >2014-08-14 20:25:10.882 30119 TRACE root return self.dispatch_method(method_sig, args, content) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method >2014-08-14 20:25:10.882 30119 TRACE root return amqp_method(self, args) >2014-08-14 20:25:10.882 30119 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close >2014-08-14 20:25:10.882 30119 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id)) >2014-08-14 20:25:10.882 30119 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_af26a7bdfe2c4e37a9f11ca437144e92' in vhost '/', (60, 20), None) >2014-08-14 20:25:10.882 30119 TRACE root
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1130315
: 926902