Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1139451

Summary: instance fails to launch: ERROR oslo.messaging._drivers.impl_rabbit [-] Failed to consume message from queue: [Errno 104] Connection reset by peer
Product: Red Hat OpenStack Reporter: wes hayutin <whayutin>
Component: openstack-novaAssignee: Russell Bryant <rbryant>
Status: CLOSED WORKSFORME QA Contact: Ami Jeain <ajeain>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.0 (RHEL 6)CC: ndipanov, sgordon, whayutin, yeylon
Target Milestone: ---Keywords: ZStream
Target Release: 5.0 (RHEL 6)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-16 17:32:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
compute log files
none
install and packstack logs none

Description wes hayutin 2014-09-09 00:00:50 UTC
Created attachment 935511 [details]
compute log files

Description of problem:

RHEL 6.5
ml2-vxlan neutron plugin
multinode setup w/ compute node

See attachment for install and log details


ERROR oslo.messaging._drivers.impl_rabbit [-] Failed to consume message from queue: [Errno 104] Connection reset by peer
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit Traceback (most recent call last):
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 622, in ensure
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     return method(*args, **kwargs)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 702, in _consume
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     return self.connection.drain_events(timeout=timeout)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/kombu/connection.py", line 281, in drain_events
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     return self.transport.drain_events(self.connection, **kwargs)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/kombu/transport/pyamqp.py", line 94, in drain_events
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     return connection.drain_events(**kwargs)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/amqp/connection.py", line 286, in drain_events
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     return amqp_method(channel, args)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/amqp/connection.py", line 489, in _close
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     self._x_close_ok()
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/amqp/connection.py", line 506, in _x_close_ok
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     self._send_method((10, 51))
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/amqp/abstract_channel.py", line 58, in _send_method
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     self.channel_id, method_sig, args, content,
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/amqp/method_framing.py", line 224, in write_method
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     write_frame(1, channel, payload)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/amqp/transport.py", line 160, in write_frame
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     pack('>BHI%dsB' % size, frame_type, channel, size, payload, 0xce),
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/eventlet/greenio.py", line 309, in sendall
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     tail = self.send(data, flags)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.6/site-packages/eventlet/greenio.py", line 295, in send
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit     total_sent += fd.send(data[total_sent:], flags)
2014-09-08 14:07:33.373 13443 TRACE oslo.messaging._drivers.impl_rabbit error: [Errno 104] Connection reset by peer

Comment 1 wes hayutin 2014-09-09 00:03:19 UTC
Created attachment 935512 [details]
install and packstack logs

Comment 3 wes hayutin 2014-09-09 00:49:32 UTC
The same setup is passing on rhel7

Comment 4 Russell Bryant 2014-10-08 15:45:22 UTC
Can you provide more information on how this occurs?  What's happening when this occurs?  Does this *never* work?  Or is this something that just happened one time?  What exactly was the failure?  Can you also provide the rabbit log?

Comment 5 wes hayutin 2014-10-08 15:59:12 UTC
All the logs should be in the attached file.
Pretty sure this is a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1144181

The recreate was to

Any distro for juno
1. install via packstack any neutron config
2. reboot
3. try to connect to neutron

which is now working in juno, Alan pushed the changes to stage, and prod already

Comment 6 wes hayutin 2014-10-08 15:59:52 UTC
one note on the dupe above.. is that this bug was opened 10 days prior to the dupe :(

Comment 7 Russell Bryant 2014-10-16 15:19:34 UTC
(In reply to wes hayutin from comment #5)
> All the logs should be in the attached file.
> Pretty sure this is a dupe of
> https://bugzilla.redhat.com/show_bug.cgi?id=1144181
> 
> The recreate was to
> 
> Any distro for juno
> 1. install via packstack any neutron config
> 2. reboot
> 3. try to connect to neutron
> 
> which is now working in juno, Alan pushed the changes to stage, and prod
> already

Are you saying it's not a problem anymore?

The dupe you mentioned seems to be unrelated to rabbit.  It's an oslo.db related error.

If the problem still exists, can you reproduce with debug logs on in openstack?