2126786 – [RHOSP 16.2] swiftclient forbids ceilometer from polling swift objects

Bug 2126786 - [RHOSP 16.2] swiftclient forbids ceilometer from polling swift objects

Summary: [RHOSP 16.2] swiftclient forbids ceilometer from polling swift objects

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-tripleo-heat-templates
Sub Component:
Version:	16.2 (Train)
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	z4
Target Release:	16.2 (Train on RHEL 8.4)
Assignee:	Yadnesh Kulkarni
QA Contact:	Leonid Natapov
Docs Contact:	Joanne O'Flynn
URL:
Whiteboard:
Depends On:	2058518 2162456
Blocks:	2130849
TreeView+	depends on / blocked

Reported:	2022-09-14 12:08 UTC by Yadnesh Kulkarni
Modified:	2023-01-19 16:09 UTC (History)
CC List:	3 users (show)
Fixed In Version:	openstack-tripleo-heat-templates-11.6.1-2.20221010235131.e0d438c.el8ost
Doc Type:	Bug Fix
Doc Text:	This update fixes a bug that prevented the Telemetry service (ceilometer) from polling the Object Storage service (swift) for metrics. The Telemetry service now polls the Object Storage service correctly.
Clone Of:
Environment:
Last Closed:	2022-12-07 19:24:42 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
OpenStack gerrit	856575	None	ABANDONED	Associate the ResellerAdmin role and ceilometer user	2022-10-03 13:53:49 UTC
OpenStack gerrit	857866	None	MERGED	Ceilometer: Ensure the ResellerAdmin role exists	2022-10-03 13:53:48 UTC
Red Hat Issue Tracker	OSP-18708	None	None	None	2022-09-14 12:15:20 UTC
Red Hat Product Errata	RHBA-2022:8794	None	None	None	2022-12-07 19:25:07 UTC

Description Yadnesh Kulkarni 2022-09-14 12:08:09 UTC

This bug was initially created as a copy of Bug #2058518

I am copying this bug because: 



Description of problem:

The following polling.yaml file
---
sources:
    - name: some_pollsters
      interval: 30
      meters:
        - cpu
        - disk.*
        - ip.*
        - image.*
        - memory
        - memory.*
        - network.*
        - perf.*
        - port
        - port.*
        - switch
        - switch.*
        - storage.*
        - volume.*

generated by following the documentation 
https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/service_telemetry_framework_1.3/assembly-completing-the-stf-configuration_assembly#creating-the-base-configuration-for-stf_assembly-completing-the-stf-configuration



causes the following errors in ceilometer/central.log

2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager [-] Continue after error from storage.objects: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden:
 swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager Traceback (most recent call last):
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/polling/manager.py", line 187, in poll_and_notify
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     for sample in samples:
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 113, in get_samples
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     cache, tenants):
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 80, in _iter_accounts
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     ksclient, tenants))
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 99, in _get_account_info
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     raise e
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 94, in _get_account_info
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     keystone_client.get_auth_token(ksclient)))
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/swiftclient/client.py", line 871, in head_account
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     raise ClientException.from_response(resp, 'Account HEAD failed', body)
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager 
2022-02-17 17:49:10.929 15 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-02-17 17:49:38.403 15 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-02-17 17:49:38.415 15 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-02-17 17:49:38.451 15 INFO ceilometer.polling.manager [-] Polling pollster image.size in the context of some_pollsters
2022-02-17 17:49:40.092 15 INFO ceilometer.polling.manager [-] Polling pollster storage.objects.containers in the context of some_pollsters
2022-02-17 17:49:40.416 15 INFO swiftclient [-] REQ: curl -i http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 -I -H "X-Auth-Token: gAAAAABiDoqy4aVP..."
2022-02-17 17:49:40.416 15 INFO swiftclient [-] RESP STATUS: 403 Forbidden
2022-02-17 17:49:40.416 15 INFO swiftclient [-] RESP HEADERS: {'Content-Length': '12', 'X-Trans-Id': 'tx0000000000000000018dc-00620e8ab4-806ad4-default', 'X-Openstack-Request-Id': 'tx0000000000000000018dc-00620e8ab
4-806ad4-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'text/plain; charset=utf-8', 'Date': 'Thu, 17 Feb 2022 17:49:40 GMT'}
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager [-] Continue after error from storage.objects.containers: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403
 Forbidden: swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager Traceback (most recent call last):
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/polling/manager.py", line 187, in poll_and_notify
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     for sample in samples:
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 149, in get_samples
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     cache, tenants):
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 80, in _iter_accounts
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     ksclient, tenants))
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 99, in _get_account_info
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     raise e
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 94, in _get_account_info
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     keystone_client.get_auth_token(ksclient)))
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/swiftclient/client.py", line 871, in head_account
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     raise ClientException.from_response(resp, 'Account HEAD failed', body)
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager 
2022-02-17 17:49:40.442 15 INFO ceilometer.polling.manager [-] Polling pollster storage.objects.size in the context of some_pollsters

Comment 7 Leonid Natapov 2022-10-31 15:22:42 UTC

No 403 Forbidden errors in the central.log file
storage pollster enabled in polling.yaml

---
sources:
    - name: some_pollsters
      interval: 30
      meters:
        - cpu
        - disk.*
        - ip.*
        - image.*
        - memory
        - memory.*
        - network.*
        - perf.*
        - port
        - port.*
        - switch
        - switch.*
        - storage.*
        - volume.*

Comment 13 errata-xmlrpc 2022-12-07 19:24:42 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.4), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8794

Note You need to log in before you can comment on or make changes to this bug.