Bug 2130849 - [RHOSP 16.1] swiftclient forbids ceilometer from polling swift objects
Summary: [RHOSP 16.1] swiftclient forbids ceilometer from polling swift objects
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z9
: 16.1 (Train on RHEL 8.2)
Assignee: Yadnesh Kulkarni
QA Contact: Leonid Natapov
Joanne O'Flynn
URL:
Whiteboard:
Depends On: 2126786
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-09-29 07:39 UTC by Yadnesh Kulkarni
Modified: 2022-12-07 20:30 UTC (History)
3 users (show)

Fixed In Version: openstack-tripleo-heat-templates-11.3.2-1.20221004053221.29a02c1.el8ost
Doc Type: Bug Fix
Doc Text:
Before this update, a Telemetry service (ceilometer) user had insufficient privileges to poll objects from the Object Storage service (swift). The Object Storage service client did not allow the Telemetry service user to fetch object details. With this update, the Telemetry service user is associated with the ResellerAdmin role. + Execute the following command to workaround this issue manually: + ---- $ openstack role add --user ceilometer --project service ResellerAdmin ---- + The associated Telemetry service user can poll Object Storage service object metrics successfully.
Clone Of:
Environment:
Last Closed: 2022-12-07 20:29:53 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 856575 0 None ABANDONED Associate the ResellerAdmin role and ceilometer user 2022-10-04 15:32:52 UTC
OpenStack gerrit 857866 0 None MERGED Ceilometer: Ensure the ResellerAdmin role exists 2022-10-04 15:32:51 UTC
Red Hat Issue Tracker OSP-18986 0 None None None 2022-09-29 07:46:20 UTC
Red Hat Product Errata RHSA-2022:8796 0 None None None 2022-12-07 20:30:09 UTC

Description Yadnesh Kulkarni 2022-09-29 07:39:18 UTC 
This bug was initially created as a copy of Bug #2126786

This bug was initially created as a copy of Bug #2058518


Description of problem:

The following polling.yaml file
---
sources:
    - name: some_pollsters
      interval: 30
      meters:
        - cpu
        - disk.*
        - ip.*
        - image.*
        - memory
        - memory.*
        - network.*
        - perf.*
        - port
        - port.*
        - switch
        - switch.*
        - storage.*
        - volume.*

generated by following the documentation 
https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/service_telemetry_framework_1.3/assembly-completing-the-stf-configuration_assembly#creating-the-base-configuration-for-stf_assembly-completing-the-stf-configuration



causes the following errors in ceilometer/central.log

2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager [-] Continue after error from storage.objects: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden:
 swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager Traceback (most recent call last):
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/polling/manager.py", line 187, in poll_and_notify
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     for sample in samples:
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 113, in get_samples
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     cache, tenants):
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 80, in _iter_accounts
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     ksclient, tenants))
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 99, in _get_account_info
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     raise e
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 94, in _get_account_info
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     keystone_client.get_auth_token(ksclient)))
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/swiftclient/client.py", line 871, in head_account
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager     raise ClientException.from_response(resp, 'Account HEAD failed', body)
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:10.917 15 ERROR ceilometer.polling.manager 
2022-02-17 17:49:10.929 15 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-02-17 17:49:38.403 15 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-02-17 17:49:38.415 15 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-02-17 17:49:38.451 15 INFO ceilometer.polling.manager [-] Polling pollster image.size in the context of some_pollsters
2022-02-17 17:49:40.092 15 INFO ceilometer.polling.manager [-] Polling pollster storage.objects.containers in the context of some_pollsters
2022-02-17 17:49:40.416 15 INFO swiftclient [-] REQ: curl -i http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 -I -H "X-Auth-Token: gAAAAABiDoqy4aVP..."
2022-02-17 17:49:40.416 15 INFO swiftclient [-] RESP STATUS: 403 Forbidden
2022-02-17 17:49:40.416 15 INFO swiftclient [-] RESP HEADERS: {'Content-Length': '12', 'X-Trans-Id': 'tx0000000000000000018dc-00620e8ab4-806ad4-default', 'X-Openstack-Request-Id': 'tx0000000000000000018dc-00620e8ab
4-806ad4-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'text/plain; charset=utf-8', 'Date': 'Thu, 17 Feb 2022 17:49:40 GMT'}
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager [-] Continue after error from storage.objects.containers: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403
 Forbidden: swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager Traceback (most recent call last):
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/polling/manager.py", line 187, in poll_and_notify
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     for sample in samples:
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 149, in get_samples
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     cache, tenants):
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 80, in _iter_accounts
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     ksclient, tenants))
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 99, in _get_account_info
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     raise e
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/ceilometer/objectstore/swift.py", line 94, in _get_account_info
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     keystone_client.get_auth_token(ksclient)))
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager   File "/usr/lib/python3.6/site-packages/swiftclient/client.py", line 871, in head_account
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager     raise ClientException.from_response(resp, 'Account HEAD failed', body)
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager swiftclient.exceptions.ClientException: Account HEAD failed: http://10.239.8.138:8080/swift/v1/AUTH_bbeb4893f55840848ad7a598a0e6dcf8 403 Forbidden
2022-02-17 17:49:40.417 15 ERROR ceilometer.polling.manager 
2022-02-17 17:49:40.442 15 INFO ceilometer.polling.manager [-] Polling pollster storage.objects.size in the context of some_pollsters
Comment 11 Leonid Natapov 2022-11-06 04:34:28 UTC 
OSP16z9 No 403 Forbidden errors in ceilometer logs

---
sources:
    - name: some_pollsters
      interval: 30
      meters:
        - cpu
        - disk.*
        - ip.*
        - image.*
        - memory
        - memory.*
        - network.*
        - perf.*
        - port
        - port.*
        - switch
        - switch.*
        - storage.*
        - volume.*
Comment 15 Jenny-Anne Lynch 2022-11-17 10:22:12 UTC 
Hi Yadnesh,

I edited the Doc Text field for 16.1.9 Release Notes. Can you review for technical accuracy and update if needed? 

Thanks,

Jenny-Anne
Comment 20 errata-xmlrpc 2022-12-07 20:29:53 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenStack 16.1.9 (openstack-tripleo-heat-templates) security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8796
Note You need to log in before you can comment on or make changes to this bug.