1348201 – Graceful handling of 404 error from neutron

Bug 1348201 - Graceful handling of 404 error from neutron

Summary: Graceful handling of 404 error from neutron

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-ceilometer
Sub Component:
Version:	9.0 (Mitaka)
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	ga
Target Release:	9.0 (Mitaka)
Assignee:	Pradeep Kilambi
QA Contact:	Yurii Prokulevych
Docs Contact:
URL:
Whiteboard:
Depends On:	1353929
Blocks:
TreeView+	depends on / blocked

Reported:	2016-06-20 12:31 UTC by Yurii Prokulevych
Modified:	2016-08-11 12:26 UTC (History)
CC List:	4 users (show)
Fixed In Version:	openstack-ceilometer-6.1.1-1.el7ost
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-08-11 12:26:54 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
OpenStack gerrit	335973	0	None	MERGED	add log decorator for neutron_client public method	2020-09-17 14:42:32 UTC
Red Hat Product Errata	RHEA-2016:1597	0	normal	SHIPPED_LIVE	Red Hat OpenStack Platform 9 Release Candidate Advisory	2016-08-11 16:06:52 UTC

Description Yurii Prokulevych 2016-06-20 12:31:35 UTC

Description of problem:
-----------------------
Default pipeline.yaml contains a wild card entry - causing central agent to poll 'all' pollsters. Though if some polled object doesn't exist or is not configured it throws traceback in log:

2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager [req-1c713065-631a-4764-a2ea-a40384fe51d9 admin - - - -] Unable to discover resources: 404 Not Found

The resource could not be found.

   
Neutron server returns request_ids: ['req-d7573c0c-4939-499c-8c30-d611791e487f']
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager Traceback (most recent call last):
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/ceilometer/agent/manager.py", line 458, in discover
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     discovered = discoverer.discover(self, param)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/ceilometer/network/services/discovery.py", line 68, in discover
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     loadbalancers = self.neutron_cli.list_loadbalancer()
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/ceilometer/neutron_client.py", line 401, in list_loadbalancer
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     resp = self.client.list_loadbalancers()
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 97, in with_params
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     ret = self.function(instance, *args, **kwargs)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1045, in list_loadbalancers
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     retrieve_all, **_params)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 373, in list
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     for r in self._pagination(collection, path, **params):
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 388, in _pagination
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     res = self.get(path, params=params)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 358, in get
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     headers=headers, params=params)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 335, in retry_request
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     headers=headers, params=params)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 298, in do_request
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     self._handle_fault_response(status_code, replybody, resp)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 273, in _handle_fault_response
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     exception_handler_v20(status_code, error_body)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 84, in exception_handler_v20
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager     request_ids=request_ids)
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager NotFound: 404 Not Found
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager 
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager The resource could not be found.
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager 
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager    
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager Neutron server returns request_ids: ['req-d7573c0c-4939-499c-8c30-d611791e487f']
2016-06-20 12:14:27.613 6075 ERROR ceilometer.agent.manager 
2016-06-20 12:14:27.614 6075 INFO ceilometer.agent.manager [req-1c713065-631a-4764-a2ea-a40384fe51d9 admin - - - -] Skip pollster network.services.lb.total.connections, no resources found this cycle
2016-06-20 12:14:27.950 6075 WARNING ceilometer.neutron_client [req-1c713065-631a-4764-a2ea-a40384fe51d9 admin - - - -] The resource could not be found.

This situation has to be handled gracefully.

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-ceilometer-api-6.0.0-2.el7ost.noarch
openstack-ceilometer-polling-6.0.0-2.el7ost.noarch
openstack-ceilometer-compute-6.0.0-2.el7ost.noarch
python-ceilometer-6.0.0-2.el7ost.noarch
openstack-ceilometer-common-6.0.0-2.el7ost.noarch
openstack-ceilometer-collector-6.0.0-2.el7ost.noarch
openstack-ceilometer-notification-6.0.0-2.el7ost.noarch
openstack-ceilometer-central-6.0.0-2.el7ost.noarch
python-ceilometerclient-2.3.0-1.el7ost.noarch

python-neutronclient-4.1.1-2.el7ost.noarch

Additional info:
----------------
Virtual setup: 3Controllers + 2Computes +1Ceph

Comment 1 Julien Danjou 2016-06-30 12:02:57 UTC

Backported upstream fix to stable/mitaka at http://review.openstack.org/335973

Comment 3 Yurii Prokulevych 2016-07-20 13:48:40 UTC

Excerpt from ceilometer/central.log:
------------------------------------
...
Skip pollster network.services.lb.total.connections, no resources found this cycle
Skip pollster network.services.lb.loadbalancer, no resources found this cycle
Skip pollster network.services.vpn.connections, no resources found this cycle
Skip pollster network.services.lb.member, no resources found this cycle

Packages:
---------
openstack-ceilometer-central-6.1.1-1.el7ost.noarch
python-ceilometerclient-2.3.0-1.el7ost.noarch
openstack-ceilometer-polling-6.1.1-1.el7ost.noarch
python-ceilometer-6.1.1-1.el7ost.noarch
openstack-ceilometer-common-6.1.1-1.el7ost.noarch
openstack-ceilometer-collector-6.1.1-1.el7ost.noarch
openstack-ceilometer-notification-6.1.1-1.el7ost.noarch
openstack-ceilometer-api-6.1.1-1.el7ost.noarch
openstack-ceilometer-compute-6.1.1-1.el7ost.noarch

Comment 5 errata-xmlrpc 2016-08-11 12:26:54 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1597.html

Note You need to log in before you can comment on or make changes to this bug.