Bug 1023747 - Fails to run instances because qpid is not running well
Fails to run instances because qpid is not running well
Status: CLOSED DUPLICATE of bug 997541
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-quantum (Show other bugs)
3.0
Unspecified Unspecified
unspecified Severity urgent
: beta
: 4.0
Assigned To: Jakub Libosvar
Ofer Blaut
: Regression, TestBlocker
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-27 11:36 EDT by Rami Vaknin
Modified: 2016-04-26 21:34 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-06 08:21:39 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Rami Vaknin 2013-10-27 11:36:16 EDT
Version
=======
Grizzly/2013-10-24.5 on rhel6.4 (does NOT reproduce on rhel6.5)
openstack-packstack-2013.1.1-0.33.dev695.el6ost
Distibuted installation.


Description
===========
Can't run instances, they all reach an "ERROR" status.
The cause is qpidd service that failed to fully start in the controller machine.


Workaroud
=========
restart the qpid service in the controller machine.


From /var/log/messages
======================

Oct 27 17:07:30 puma10 qpidd[2293]: 2013-10-27 17:07:30 error Execution exception: not-found: Exchange type not implemented: Direct (qpid/broker/SessionAdapter.cpp:117)
Oct 27 17:07:30 puma10 qpidd[2293]: 2013-10-27 17:07:30 error Execution exception: not-found: Exchange type not implemented: Direct (qpid/broker/SessionAdapter.cpp:117)
Oct 27 17:07:30 puma10 qpidd[2293]: 2013-10-27 17:07:30 error Execution exception: not-found: Exchange type not implemented: Direct (qpid/broker/SessionAdapter.cpp:117)
Oct 27 17:07:30 puma10 qpidd[2293]: 2013-10-27 17:07:30 error Execution exception: not-found: Exchange type not implemented: Direct (qpid/broker/SessionAdapter.cpp:117)


$ rpm -qa | grep qpid
qpid-cpp-client-0.14-22.el6_3.x86_64
qpid-cpp-server-0.14-22.el6_3.x86_64
python-qpid-0.14-11.el6_3.noarch


# nova show cb572664-acd2-4c73-a0dc-69c243feb7e8
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------+
| Property                            | Value                                                                                                           |
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------+
| status                              | ERROR                                                                                                           |
| updated                             | 2013-10-27T13:30:44Z                                                                                            |
| OS-EXT-STS:task_state               | None                                                                                                            |
| OS-EXT-SRV-ATTR:host                | puma07.scl.lab.tlv.redhat.com                                                                                   |
| key_name                            | oskey                                                                                                           |
| image                               | rhel64 (1f86c6dd-0a62-45f6-92cb-160a52030005)                                                                   |
| hostId                              | 132515c0176ef16e02c71d56122f78f5eff90f9dc6083372fec6fb40                                                        |
| OS-EXT-STS:vm_state                 | error                                                                                                           |
| OS-EXT-SRV-ATTR:instance_name       | instance-00000001                                                                                               |
| OS-EXT-SRV-ATTR:hypervisor_hostname | puma07.scl.lab.tlv.redhat.com                                                                                   |
| flavor                              | m1.small (2)                                                                                                    |
| id                                  | cb572664-acd2-4c73-a0dc-69c243feb7e8                                                                            |
| user_id                             | cc06aa239ead4a5896d9012646eb80fc                                                                                |
| name                                | vlan_211_0                                                                                                      |
| created                             | 2013-10-27T13:30:42Z                                                                                            |
| tenant_id                           | 72f3908b58034bed943033a918d3e20d                                                                                |
| OS-DCF:diskConfig                   | MANUAL                                                                                                          |
| metadata                            | {}                                                                                                              |
| accessIPv4                          |                                                                                                                 |
| accessIPv6                          |                                                                                                                 |
| fault                               | {u'message': u'QuantumClientException', u'code': 500, u'details': u'[Errno 113] EHOSTUNREACH                    |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 224, in decorated_function              |
|                                     |     return function(self, context, *args, **kwargs)                                                             |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1240, in run_instance                   |
|                                     |     do_run_instance()                                                                                           |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 242, in inner                |
|                                     |     retval = f(*args, **kwargs)                                                                                 |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1239, in do_run_instance                |
|                                     |     admin_password, is_first_time, node, instance)                                                              |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 895, in _run_instance                   |
|                                     |     self._set_instance_error_state(context, instance[\'uuid\'])                                                 |
|                                     |   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__                                               |
|                                     |     self.gen.next()                                                                                             |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 887, in _run_instance                   |
|                                     |     filter_properties, bdms)                                                                                    |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 927, in _reschedule_or_reraise          |
|                                     |     self._log_original_error(exc_info, instance_uuid)                                                           |
|                                     |   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__                                               |
|                                     |     self.gen.next()                                                                                             |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 922, in _reschedule_or_reraise          |
|                                     |     self._shutdown_instance(context, instance, bdms)                                                            |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1262, in _shutdown_instance             |
|                                     |     network_info = self._get_instance_nw_info(context, instance)                                                |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 697, in _get_instance_nw_info           |
|                                     |     instance, conductor_api=self.conductor_api)                                                                 |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/network/quantumv2/api.py", line 368, in get_instance_nw_info      |
|                                     |     result = self._get_instance_nw_info(context, instance, networks)                                            |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/network/quantumv2/api.py", line 376, in _get_instance_nw_info     |
|                                     |     nw_info = self._build_network_info_model(context, instance, networks)                                       |
|                                     |   File "/usr/lib/python2.6/site-packages/nova/network/quantumv2/api.py", line 795, in _build_network_info_model |
|                                     |     data = client.list_ports(**search_opts)                                                                     |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 107, in with_params                |
|                                     |     ret = self.function(instance, *args, **kwargs)                                                              |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 255, in list_ports                 |
|                                     |     **_params)                                                                                                  |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 996, in list                       |
|                                     |     for r in self._pagination(collection, path, **params):                                                      |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 1009, in _pagination               |
|                                     |     res = self.get(path, params=params)                                                                         |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 982, in get                        |
|                                     |     headers=headers, params=params)                                                                             |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 967, in retry_request              |
|                                     |     headers=headers, params=params)                                                                             |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 912, in do_request                 |
|                                     |     self._handle_fault_response(status_code, replybody)                                                         |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 893, in _handle_fault_response     |
|                                     |     exception_handler_v20(status_code, des_error_body)                                                          |
|                                     |   File "/usr/lib/python2.6/site-packages/quantumclient/v2_0/client.py", line 87, in exception_handler_v20       |
|                                     |     message=message)                                                                                            |
|                                     | ', u'created': u'2013-10-27T13:30:44Z'}                                                                         |
| OS-EXT-STS:power_state              | 0                                                                                                               |
| OS-EXT-AZ:availability_zone         | nova                                                                                                            |
| config_drive                        |                                                                                                                 |
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------+
Comment 8 Jakub Libosvar 2013-11-06 08:21:39 EST

*** This bug has been marked as a duplicate of bug 997541 ***

Note You need to log in before you can comment on or make changes to this bug.