Bug 2058518

@mrunge any suggestions on how to adjust this? Looks like it's trying to authenticate against an API that maybe doesn't exist? Seems like the configuration is correct, but that the backend systems aren't available for polling.

https://github.com/openstack/ceilometer/blob/master/etc/ceilometer/polling.yaml

Assigning 'ResellerAdmin' role to ceilometer user seems to fix this issue
~~~
$ openstack role add --user ceilometer --project service ResellerAdmin
~~~

After this ceilometer could poll swift containers of all projects
~~~
2022-07-27 13:58:41.605 15 DEBUG swiftclient [-] RESP BODY: b'[{"name": "testcont", "count": 0, "bytes": 0, "last_modified": "2022-07-26T10:04:29.069490"}]' http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:163
2022-07-27 13:58:41.618 15 DEBUG swiftclient [-] REQ: curl -i None?format=json -X GET -H "X-Auth-Token: gAAAAABi4USRGuiW..." -H "Accept-Encoding: gzip" http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:157
2022-07-27 13:58:41.618 15 DEBUG swiftclient [-] RESP STATUS: 200 OK http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:158
2022-07-27 13:58:41.618 15 DEBUG swiftclient [-] RESP HEADERS: {'content-type': 'application/json; charset=utf-8', 'x-account-container-count': '1', 'x-account-object-count': '2', 'x-account-bytes-used': '32677888', 'x-timestamp': '1658818296.07454', 'x-account-storage-policy-policy-0-container-count': '1', 'x-account-storage-policy-policy-0-object-count': '2', 'x-account-storage-policy-policy-0-bytes-used': '32677888', 'accept-ranges': 'bytes', 'content-length': '98', 'x-account-project-domain-id': 'default', 'x-trans-id': 'txec7ccf4dff924a588e2e1-0062e14491', 'x-openstack-request-id': 'txec7ccf4dff924a588e2e1-0062e14491', 'date': 'Wed, 27 Jul 2022 13:58:41 GMT'} http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:159

2022-07-27 13:58:41.618 15 DEBUG swiftclient [-] RESP BODY: b'[{"name": "glance", "count": 2, "bytes": 32677888, "last_modified": "2022-07-26T06:51:36.138870"}]' http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:163
2022-07-27 13:58:41.629 15 DEBUG swiftclient [-] REQ: curl -i None?format=json -X GET -H "X-Auth-Token: gAAAAABi4USRGuiW..." -H "Accept-Encoding: gzip" http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:157
2022-07-27 13:58:41.629 15 DEBUG swiftclient [-] RESP STATUS: 200 OK http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:158
2022-07-27 13:58:41.629 15 DEBUG swiftclient [-] RESP HEADERS: {'content-type': 'application/json; charset=utf-8', 'content-length': '2', 'x-account-container-count': '0', 'x-account-object-count': '0', 'x-account-bytes-used': '0', 'x-timestamp': '1658930321.62755', 'x-put-timestamp': '1658930321.62755', 'x-trans-id': 'txa96ed7a03940417d9ae94-0062e14491', 'x-openstack-request-id': 'txa96ed7a03940417d9ae94-0062e14491', 'date': 'Wed, 27 Jul 2022 13:58:41 GMT'} http_log /usr/lib/python3.9/site-packages/swiftclient/client.py:159
~~~

Fix merged upstream master https://review.opendev.org/c/openstack/tripleo-heat-templates/+/854033

Forbidden messages appear in OSP17.0.1
openstack-tripleo-heat-templates-14.3.1-0.20221208160327.feca772.el9ost.noarch

polling.yaml
------------

[root@controller-0 ~]# cat /var/lib/config-data/puppet-generated/ceilometer/etc/ceilometer/polling.yaml
---
sources:
    - name: some_pollsters
      interval: 30
      meters:
        - cpu
        - disk.*
        - ip.*
        - image.*
        - memory
        - memory.*
        - network.*
        - perf.*
        - port
        - port.*
        - switch
        - switch.*
        - storage.*
        - volume.*
-------------------------

2023-01-17 09:01:03.863 14 ERROR ceilometer.polling.manager 
2023-01-17 09:01:03.864 14 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.outgoing.bytes, no  resources found this cycle poll_and_notify /usr/lib/python3.9/site-packages/ceilometer/polling/manager.py:177
2023-01-17 09:01:03.930 14 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2023-01-17 09:01:03.931 14 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.vpn.connections, no  resources found this cycle poll_and_notify /usr/lib/python3.9/site-packages/ceilometer/polling/manager.py:177
2023-01-17 09:01:03.931 14 DEBUG ceilometer.polling.manager [-] Skip pollster switch.port.transmit.packets, no  resources found this cycle poll_and_notify /usr/lib/python3.9/site-packages/ceilometer/polling/manager.py:177
2023-01-17 09:01:03.931 14 INFO ceilometer.polling.manager [-] Polling pollster storage.objects in the context of some_pollsters
2023-01-17 09:01:03.984 14 INFO swiftclient [-] REQ: curl -i None -I -H "X-Auth-Token: gAAAAABjxmPOS9uQ..."
2023-01-17 09:01:03.984 14 INFO swiftclient [-] RESP STATUS: 403 Forbidden
2023-01-17 09:01:03.985 14 INFO swiftclient [-] RESP HEADERS: {'content-length': '12', 'x-trans-id': 'tx000009c427ba0b07d8b5d-0063c663cf-5ef6-default', 'x-openstack-request-id': 'tx000009c427ba0b07d8b5d-0063c663cf-5ef6-default', 'accept-ranges':
 'bytes', 'content-type': 'text/plain; charset=utf-8', 'date': 'Tue, 17 Jan 2023 09:01:03 GMT'}
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager [-] Continue after error from storage.objects: Account HEAD failed: http://172.17.3.95:8080/swift/v1/AUTH_435189f702f5412994c086c6567eee86 403 Forbidden (txn: tx000009c427ba0b07d8b5d-00
63c663cf-5ef6-default): swiftclient.exceptions.ClientException: Account HEAD failed: http://172.17.3.95:8080/swift/v1/AUTH_435189f702f5412994c086c6567eee86 403 Forbidden (txn: tx000009c427ba0b07d8b5d-0063c663cf-5ef6-default)
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager Traceback (most recent call last):
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager   File "/usr/lib/python3.9/site-packages/ceilometer/polling/manager.py", line 194, in poll_and_notify
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager     for sample in samples:
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager   File "/usr/lib/python3.9/site-packages/ceilometer/objectstore/swift.py", line 114, in get_samples
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager     for tenant, account in self._iter_accounts(manager.keystone,
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager   File "/usr/lib/python3.9/site-packages/ceilometer/objectstore/swift.py", line 77, in _iter_accounts
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager     cache[self.CACHE_KEY_METHOD] = list(self._get_account_info(
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager   File "/usr/lib/python3.9/site-packages/ceilometer/objectstore/swift.py", line 101, in _get_account_info
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager     raise e
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager   File "/usr/lib/python3.9/site-packages/ceilometer/objectstore/swift.py", line 93, in _get_account_info
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager     yield (t.id, swift_api_method(
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager   File "/usr/lib/python3.9/site-packages/swiftclient/client.py", line 924, in head_account
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager     raise ClientException.from_response(resp, 'Account HEAD failed', body)
2023-01-17 09:01:03.985 14 ERROR ceilometer.polling.manager swiftclient.exceptions.ClientException: Account HEAD failed: http://172.17.3.95:8080/swift/v1/AUTH_435189f702f5412994c086c6567eee86 403 Forbidden (txn: tx000009c427ba0b07d8b5d-0063c663c
f-5ef6-default)