Bug 1512880 - openstack metric event list gives an error in undercloud
Summary: openstack metric event list gives an error in undercloud
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: rc
: 12.0 (Pike)
Assignee: Pradeep Kilambi
QA Contact: Sasha Smolyak
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-14 11:52 UTC by Sasha Smolyak
Modified: 2018-02-05 19:15 UTC (History)
9 users (show)

Fixed In Version: instack-undercloud-7.4.3-3.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-13 22:20:31 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1732459 0 None None None 2017-11-15 16:15:54 UTC
OpenStack gerrit 520087 0 None MERGED Fix panko ssl port 2020-07-15 04:23:51 UTC
OpenStack gerrit 520116 0 None MERGED Fix panko ssl port 2020-07-15 04:23:51 UTC
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Sasha Smolyak 2017-11-14 11:52:31 UTC
Description of problem:
openstack metric event list in undercloud gives an error:
Unable to establish connection to https://192.168.24.2:13779/v2/events: HTTPSConnectionPool(host='192.168.24.2', port=13779): Max retries exceeded with url: /v2/events (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x41139d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

openstack metric event type list:
Unable to establish connection to https://192.168.24.2:13779/v2/event_types/: HTTPSConnectionPool(host='192.168.24.2', port=13779): Max retries exceeded with url: /v2/event_types/ (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x37917d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

in overcloud works ok.

In The logs:
(undercloud) [stack@undercloud-0 ~]$ openstack server list
+--------------------------------------+--------------+--------+------------------------+----------------+------------+
| ID                                   | Name         | Status | Networks               | Image          | Flavor     |
+--------------------------------------+--------------+--------+------------------------+----------------+------------+
| 3ed4fd57-abd0-406c-a7e1-2ed1c34265ca | compute-0    | ACTIVE | ctlplane=192.168.24.23 | overcloud-full | compute    |
| eb66f2ca-0a18-4fda-8a17-d64ea7f8f22c | ceph-1       | ACTIVE | ctlplane=192.168.24.16 | overcloud-full | ceph       |
| 0aebe045-4e08-432f-ae5b-14ef5ea1f629 | ceph-0       | ACTIVE | ctlplane=192.168.24.11 | overcloud-full | ceph       |
| 1d99f9c9-912e-480d-8b8e-6e6792dad00d | controller-2 | ACTIVE | ctlplane=192.168.24.18 | overcloud-full | controller |
| 02831378-0395-407d-9856-b8ac287552bd | controller-0 | ACTIVE | ctlplane=192.168.24.21 | overcloud-full | controller |
| f46ab971-2fe7-4766-aee3-cb4aca46f26a | controller-1 | ACTIVE | ctlplane=192.168.24.12 | overcloud-full | controller |
| 11569104-9053-4568-a16b-f2abd258d7a2 | ceph-2       | ACTIVE | ctlplane=192.168.24.20 | overcloud-full | ceph       |
| 369ba8fe-d019-43c6-bff1-74905f4d4d63 | compute-1    | ACTIVE | ctlplane=192.168.24.13 | overcloud-full | compute    |
+--------------------------------------+--------------+--------+------------------------+----------------+------------+
(undercloud) [stack@undercloud-0 ~]$ sudo tail -n 20 /var/log/ceilometer/agent-notification.log
2017-11-14 06:39:46.068 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 02831378-0395-407d-9856-b8ac287552bd does not exist (HTTP 404): ResourceNotFound: Resource 02831378-0395-407d-9856-b8ac287552bd does not exist (HTTP 404)
2017-11-14 06:39:46.091 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 369ba8fe-d019-43c6-bff1-74905f4d4d63 does not exist (HTTP 404): ResourceNotFound: Resource 369ba8fe-d019-43c6-bff1-74905f4d4d63 does not exist (HTTP 404)
2017-11-14 06:39:46.463 2802 DEBUG ceilometer.dispatcher.gnocchi [-] 8 measures posted against 8 metrics through 8 resources batch_measures /usr/lib/python2.7/site-packages/ceilometer/dispatcher/gnocchi.py:417
2017-11-14 06:39:46.489 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 11569104-9053-4568-a16b-f2abd258d7a2 does not exist (HTTP 404): ResourceNotFound: Resource 11569104-9053-4568-a16b-f2abd258d7a2 does not exist (HTTP 404)
2017-11-14 06:39:46.513 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 3ed4fd57-abd0-406c-a7e1-2ed1c34265ca does not exist (HTTP 404): ResourceNotFound: Resource 3ed4fd57-abd0-406c-a7e1-2ed1c34265ca does not exist (HTTP 404)
2017-11-14 06:39:46.536 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource eb66f2ca-0a18-4fda-8a17-d64ea7f8f22c does not exist (HTTP 404): ResourceNotFound: Resource eb66f2ca-0a18-4fda-8a17-d64ea7f8f22c does not exist (HTTP 404)
2017-11-14 06:39:46.561 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 1d99f9c9-912e-480d-8b8e-6e6792dad00d does not exist (HTTP 404): ResourceNotFound: Resource 1d99f9c9-912e-480d-8b8e-6e6792dad00d does not exist (HTTP 404)
2017-11-14 06:39:46.586 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource f46ab971-2fe7-4766-aee3-cb4aca46f26a does not exist (HTTP 404): ResourceNotFound: Resource f46ab971-2fe7-4766-aee3-cb4aca46f26a does not exist (HTTP 404)
2017-11-14 06:39:46.609 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 0aebe045-4e08-432f-ae5b-14ef5ea1f629 does not exist (HTTP 404): ResourceNotFound: Resource 0aebe045-4e08-432f-ae5b-14ef5ea1f629 does not exist (HTTP 404)
2017-11-14 06:39:46.635 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 02831378-0395-407d-9856-b8ac287552bd does not exist (HTTP 404): ResourceNotFound: Resource 02831378-0395-407d-9856-b8ac287552bd does not exist (HTTP 404)
2017-11-14 06:39:46.658 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 369ba8fe-d019-43c6-bff1-74905f4d4d63 does not exist (HTTP 404): ResourceNotFound: Resource 369ba8fe-d019-43c6-bff1-74905f4d4d63 does not exist (HTTP 404)
2017-11-14 06:39:47.039 2802 DEBUG ceilometer.dispatcher.gnocchi [-] 8 measures posted against 8 metrics through 8 resources batch_measures /usr/lib/python2.7/site-packages/ceilometer/dispatcher/gnocchi.py:417
2017-11-14 06:39:47.064 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 11569104-9053-4568-a16b-f2abd258d7a2 does not exist (HTTP 404): ResourceNotFound: Resource 11569104-9053-4568-a16b-f2abd258d7a2 does not exist (HTTP 404)
2017-11-14 06:39:47.087 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 3ed4fd57-abd0-406c-a7e1-2ed1c34265ca does not exist (HTTP 404): ResourceNotFound: Resource 3ed4fd57-abd0-406c-a7e1-2ed1c34265ca does not exist (HTTP 404)
2017-11-14 06:39:47.110 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource eb66f2ca-0a18-4fda-8a17-d64ea7f8f22c does not exist (HTTP 404): ResourceNotFound: Resource eb66f2ca-0a18-4fda-8a17-d64ea7f8f22c does not exist (HTTP 404)
2017-11-14 06:39:47.134 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 1d99f9c9-912e-480d-8b8e-6e6792dad00d does not exist (HTTP 404): ResourceNotFound: Resource 1d99f9c9-912e-480d-8b8e-6e6792dad00d does not exist (HTTP 404)
2017-11-14 06:39:47.157 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource f46ab971-2fe7-4766-aee3-cb4aca46f26a does not exist (HTTP 404): ResourceNotFound: Resource f46ab971-2fe7-4766-aee3-cb4aca46f26a does not exist (HTTP 404)
2017-11-14 06:39:47.182 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 0aebe045-4e08-432f-ae5b-14ef5ea1f629 does not exist (HTTP 404): ResourceNotFound: Resource 0aebe045-4e08-432f-ae5b-14ef5ea1f629 does not exist (HTTP 404)
2017-11-14 06:39:47.208 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 02831378-0395-407d-9856-b8ac287552bd does not exist (HTTP 404): ResourceNotFound: Resource 02831378-0395-407d-9856-b8ac287552bd does not exist (HTTP 404)
2017-11-14 06:39:47.232 2802 ERROR ceilometer.dispatcher.gnocchi [-] Resource 369ba8fe-d019-43c6-bff1-74905f4d4d63 does not exist (HTTP 404): ResourceNotFound: Resource 369ba8fe-d019-43c6-bff1-74905f4d4d63 does not exist (HTTP 404)

Version-Release number of selected component (if applicable):
openstack-ceilometer-notification-9.0.1-3.el7ost.noarch
openstack-ceilometer-central-9.0.1-3.el7ost.noarch
python-ceilometermiddleware-1.1.0-1.el7ost.noarch
python-ceilometerclient-2.9.0-1.el7ost.noarch
puppet-ceilometer-11.3.1-0.20170923033230.767c543.el7ost.noarch
openstack-ceilometer-common-9.0.1-3.el7ost.noarch
openstack-ceilometer-polling-9.0.1-3.el7ost.noarch
python-ceilometer-9.0.1-3.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deployed new setup. Performed manual update: 
sudo yum localinstall -y http://rhos-release.virt.bos.redhat.com/repos/rhos-release/rhos-release-latest.noarch.rpm

sudo yum localinstall -y http://download.lab.bos.redhat.com/rcm-guest/puddles/OpenStack/rhos-release/rhos-release-latest.noarch.rpm # alternative location

http://download.eng.bos.redhat.com/brewroot/packages/wget/1.14/15.el7/x86_64/wget-1.14-15.el7.x86_64.rpm

sudo rhos-release -x
sudo rhos-release -P 12
sudo yum update python-tripleoclient
su - stack openstack undercloud upgrade 2>&1 | tee undercloud_update.log


2. Tried to check events list, got the error
3.Watched the agent-notification.log, saw that the error was there from the beginning

Actual results:
error for openstack metric event type list and openstack metric event list

Expected results:
The lists

Additional info:

Comment 1 Sasha Smolyak 2017-11-15 14:00:28 UTC
Happens straight after deployment, without updates. Event list in undercloud is unreachable

Comment 2 Red Hat Bugzilla Rules Engine 2017-11-15 14:16:44 UTC
This bugzilla has been removed from the release since it has not been triaged, and needs to be reviewed for targeting another release.

Comment 6 Sasha Smolyak 2017-11-23 13:54:00 UTC
Verified, 
openstack event list works OK in undercloud

Comment 9 errata-xmlrpc 2017-12-13 22:20:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462


Note You need to log in before you can comment on or make changes to this bug.