Bug 1384005 - ceilometer-api fails to start on undercloud - Address already in use
Summary: ceilometer-api fails to start on undercloud - Address already in use
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ceilometer
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: rc
: 10.0 (Newton)
Assignee: Pradeep Kilambi
QA Contact: Sasha Smolyak
URL:
Whiteboard:
Depends On: 1384927
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-10-12 10:48 UTC by Yurii Prokulevych
Modified: 2018-05-14 05:25 UTC (History)
12 users (show)

Fixed In Version: openstack-ceilometer-7.0.0-2.1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-14 22:00:21 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2948 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 19:55:27 UTC
Launchpad 1632682 None None None 2016-10-12 12:29:27 UTC

Description Yurii Prokulevych 2016-10-12 10:48:15 UTC
Description of problem:
-----------------------
Ceilometer api fails to start on undercloud. Stracing the startup shows next error:

write(3, "2016-10-12 06:32:12.642 23995 CRITICAL ceilometer [-] error: [Errno 98] Address already in use\n2016-10-12 06:32:12.642 23995 ERROR ceilometer Traceback (most recent call last):\n2016-10-12 06:32:12.642 23995 ERROR ceilometer   File \"/usr/bin/ceilometer-api\", line 20, in <module>\n2016-10-12 06:32:12.642 23995 ERROR ceilometer     server = wss.make_server('', args.port, build_wsgi_app())\n2016-10-12 06:32:12.642 23995 ERROR ceilometer   File \"/usr/lib64/python2.7/wsgiref/simple_server.py\", line 144, in make_server\n2016-10-12 06:32:12.642 23995 ERROR ceilometer     server = server_class((host, port), handler_class)\n2016-10-12 06:32:12.642 23995 ERROR ceilometer   File \"/usr/lib64/python2.7/SocketServer.py\", line 419, in __init__\n2016-10-12 06:32:12.642 23995 ERROR ceilometer     self.server_bind()\n2016-10-12 06:32:12.642 23995 ERROR ceilometer   File \"/usr/lib64/python2.7/wsgiref/simple_server.py\", line 48, in server_bind\n2016-10-12 06:32:12.642 23995 ERROR ceilometer     HTTPServer.server_bind(self)\n2016-10-12 06:32:12.642 23995 ERROR ceilometer   File \"/usr/lib64/python2.7/BaseHTTPServer.py\", line 108, in server_bind\n2016-10-12 06:32:12.642 23995 ERROR ceilometer     SocketServer.TCPServer.server_bind(self)\n2016-10-12 06:32:12.642 23995 ERROR ceilometer   File \"/usr/lib64/python2.7/SocketServer.py\", line 430, in server_bind\n2016-10-12 06:32:12.642 23995 ERROR ceilometer     self.socket.bind(self.server_address)\n2016-10-12 06:32:12.642 23995 ERROR ceilometer   File \"/usr/lib64/python2.7/socket.py\", line 224, in meth\n2016-10-12 06:32:12.642 23995 ERROR ceilometer     return getattr(self._sock,name)(*args)\n2016-10-12 06:32:12.642 23995 ERROR ceilometer error: [Errno 98] Address already in use\n2016-10-12 06:32:12.642 23995 ERROR ceilometer \n", 1764) = 1764

File /usr/bin/ceilometer-api has default value for port set to 8000.

And indeed, heat-api-cfn is listening on that port:

ss -anp | grep 8000
tcp    LISTEN     0      128    192.0.2.1:8000                  *:*                   users:(("heat-api-cfn",pid=22048,fd=4))

Changing default port to 8777 resolved the issue.


Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-ceilometer-central-7.0.0-0.20160928024313.67bbd3f.el7ost.noarch
puppet-ceilometer-9.4.0-1.el7ost.noarch
openstack-ceilometer-common-7.0.0-0.20160928024313.67bbd3f.el7ost.noarch
openstack-ceilometer-collector-7.0.0-0.20160928024313.67bbd3f.el7ost.noarch
openstack-ceilometer-notification-7.0.0-0.20160928024313.67bbd3f.el7ost.noarch
python-ceilometerclient-2.6.1-1.el7ost.noarch
openstack-ceilometer-polling-7.0.0-0.20160928024313.67bbd3f.el7ost.noarch
python-ceilometer-7.0.0-0.20160928024313.67bbd3f.el7ost.noarch
openstack-ceilometer-api-7.0.0-0.20160928024313.67bbd3f.el7ost.noarch

openstack-tripleo-heat-templates-5.0.0-0.20161003064637.d636e3a.1.1.el7ost.noarch

How reproducible:
-----------------
100%

Steps to Reproduce:
1. Install telemetry on undercloud by enabling it in undercloud.conf:
    enable_telemetry = True
2. Start openstack-ceilometer-api


Actual results:
---------------
Service fails to start


Expected results:
-----------------
Service is started

Comment 1 Marios Andreou 2016-10-12 12:17:42 UTC
this should be set to 8777 by the instack install https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L653

Can you clarify what steps were taken here - was this done as part of the undercloud upgrade? Or did you just set 'enable_telemetry' and start the ceilometer api? You need to run the instack undercloud install (e.g. as part of upgrade) for the enable_telemetry to take effect.

Comment 3 Pradeep Kilambi 2016-10-12 12:28:28 UTC
This is an upstream issue looks like. Lets open a bug in launchpad and align it to ceilometer.

Comment 4 Julien Danjou 2016-10-12 12:40:22 UTC
ceilometer-api is generaeted using pbr (the same goes for aodh-api, heat-*-api, gnocchi-api, etc) and is not intended to be used outside of developement environment. There's no control on the default port which is hardcoded in pbr.

To specify a custom port, the init script should pass the --port option to ceilometer-api.

Comment 6 Julien Danjou 2016-10-12 14:03:20 UTC
Eventlet has been removed cycles ago, so what runs is a mono-thread mono-process version of Ceilometer which is going to be barely usable.

Are we forced to run Ceilometer API on undercloud?

Comment 13 errata-xmlrpc 2016-12-14 16:16:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html


Note You need to log in before you can comment on or make changes to this bug.