Bug 2294700
| Summary: | [rgw][ssl][s3-tests]: sts tests failed with access denied for list buckets of alt user in teardown phase | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Hemanth Sai <hmaheswa> |
| Component: | RGW | Assignee: | Pritha Srivastava <prsrivas> |
| Status: | CLOSED ERRATA | QA Contact: | Hemanth Sai <hmaheswa> |
| Severity: | high | Docs Contact: | Rivka Pollack <rpollack> |
| Priority: | unspecified | ||
| Version: | 7.1 | CC: | ceph-eng-bugs, cephqe-warriors, mbenjamin, prsrivas, rpollack, tserlin |
| Target Milestone: | --- | ||
| Target Release: | 9.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | ceph-20.1.0-26 | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2026-01-29 06:48:26 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Red Hat Ceph Storage 9.0 Security and Enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2026:1536 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days or the product is inactive and locked |
Description of problem: below sts tests failed with access denied for list buckets of alt user in teardown phase. this issue is seen with rgw ssl. 2024-06-27 23:30:23,893 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'FAILED s3tests_boto3/functional/test_sts.' 2024-06-27 23:30:23,893 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'py::test_get_session_token - botocor...' 2024-06-27 23:30:23,893 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'FAILED s3tests_boto3/functional/test_sts.py::test_get_session_token_permanent_creds_denied' 2024-06-27 23:30:23,894 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'FAILED s3tests_boto3/functional/test_sts.py::test_assume_role_allow - botocor...' 2024-06-27 23:30:23,894 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'FAILED s3tests_boto3/functional/test_sts.py::test_assume_role_deny - botocore...' 2024-06-27 23:30:23,894 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'FAILED s3tests_boto3/functional/test_sts.py::test_assume_role_creds_expiry - ...' 2024-06-27 23:30:23,894 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'FAILED s3tests_boto3/functional/test_sts.py::test_assume_role_deny_head_nonexistent' 2024-06-27 23:30:23,894 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'FAILED s3tests_boto3/functional/test_sts.py::test_assume_role_allow_head_nonexistent' 2024-06-27 23:30:23,894 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'ERROR s3tests_boto3/functional/test_sts.py::test_get_session_token_permanent_creds_denied' 2024-06-27 23:30:23,894 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'ERROR s3tests_boto3/functional/test_sts.py::test_assume_role_allow - botocore...' 2024-06-27 23:30:23,895 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'ERROR s3tests_boto3/functional/test_sts.py::test_assume_role_deny - botocore....' 2024-06-27 23:30:23,895 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'ERROR s3tests_boto3/functional/test_sts.py::test_assume_role_creds_expiry - b...' 2024-06-27 23:30:23,895 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'ERROR s3tests_boto3/functional/test_sts.py::test_assume_role_deny_head_nonexistent' 2024-06-27 23:30:23,895 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'ERROR s3tests_boto3/functional/test_sts.py::test_assume_role_allo' 2024-06-27 23:30:23,895 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'w_head_nonexistent' 2024-06-27 23:30:22,019 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'' 2024-06-27 23:30:22,020 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'==================================== ERRORS ====================================' 2024-06-27 23:30:22,020 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'______ ERROR at teardown of test_get_session_token_permanent_creds_denied ______' 2024-06-27 23:30:22,021 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'' 2024-06-27 23:30:22,021 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b"configfile = <class 'munch.Munch'>" 2024-06-27 23:30:22,021 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'' 2024-06-27 23:30:22,022 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' @pytest.fixture(autouse=True)' 2024-06-27 23:30:22,022 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' def setup_teardown(configfile):' 2024-06-27 23:30:22,023 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' yield' 2024-06-27 23:30:22,023 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'> teardown()' 2024-06-27 23:30:22,023 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'' 2024-06-27 23:30:22,024 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b's3tests_boto3/functional/__init__.py:315: ' 2024-06-27 23:30:22,024 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ' 2024-06-27 23:30:22,024 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b's3tests_boto3/functional/__init__.py:286: in teardown' 2024-06-27 23:30:22,025 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' nuke_prefixed_buckets(prefix=prefix, client=alt_client)' 2024-06-27 23:30:22,025 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b's3tests_boto3/functional/__init__.py:145: in nuke_prefixed_buckets' 2024-06-27 23:30:22,026 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' buckets = get_buckets_list(client, prefix)' 2024-06-27 23:30:22,026 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b's3tests_boto3/functional/__init__.py:56: in get_buckets_list' 2024-06-27 23:30:22,026 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' response = client.list_buckets()' 2024-06-27 23:30:22,027 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call' 2024-06-27 23:30:22,027 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' return self._make_api_call(operation_name, kwargs)' 2024-06-27 23:30:22,027 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ' 2024-06-27 23:30:22,028 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'_ _ _ _ _ _ _ _ _ ' 2024-06-27 23:30:22,028 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'' 2024-06-27 23:30:22,028 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'self = <botocore.client.S3 object at 0x7ff14fd0f670>' 2024-06-27 23:30:22,029 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b"operation_name = 'ListBuckets', api_params = {}" 2024-06-27 23:30:22,029 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'' 2024-06-27 23:30:22,030 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' def _make_api_call(self, operation_name, api_params):' 2024-06-27 23:30:22,030 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' operation_model = self._service_model.operation_model(operation_name)' 2024-06-27 23:30:22,030 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' service_name = self._service_model.service_name' 2024-06-27 23:30:22,030 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' history_recorder.record(' 2024-06-27 23:30:22,030 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'API_CALL'," 2024-06-27 23:30:22,030 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' {' 2024-06-27 23:30:22,031 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'service': service_name," 2024-06-27 23:30:22,031 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'operation': operation_name," 2024-06-27 23:30:22,031 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'params': api_params," 2024-06-27 23:30:22,031 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' },' 2024-06-27 23:30:22,031 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' )' 2024-06-27 23:30:22,032 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' if operation_model.deprecated:' 2024-06-27 23:30:22,032 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' logger.debug(' 2024-06-27 23:30:22,032 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'Warning: %s.%s() is deprecated', service_name, operation_name" 2024-06-27 23:30:22,032 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' )' 2024-06-27 23:30:22,032 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' request_context = {' 2024-06-27 23:30:22,033 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'client_region': self.meta.region_name," 2024-06-27 23:30:22,033 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'client_config': self.meta.config," 2024-06-27 23:30:22,033 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'has_streaming_input': operation_model.has_streaming_input," 2024-06-27 23:30:22,033 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'auth_type': operation_model.auth_type," 2024-06-27 23:30:22,033 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' }' 2024-06-27 23:30:22,034 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' request_dict = self._convert_to_request_dict(' 2024-06-27 23:30:22,034 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' api_params, operation_model, context=request_context' 2024-06-27 23:30:22,034 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' )' 2024-06-27 23:30:22,034 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' resolve' 2024-06-27 23:30:22,034 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'_checksum_context(request_dict, operation_model, api_params)' 2024-06-27 23:30:22,035 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' ' 2024-06-27 23:30:22,035 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' service_id = self._service_model.service_id.hyphenize()' 2024-06-27 23:30:22,035 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' handler, event_response = self.meta.events.emit_until_response(' 2024-06-27 23:30:22,035 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'before-call.{service_id}.{operation_name}'.format(" 2024-06-27 23:30:22,035 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' service_id=service_id, operation_name=operation_name' 2024-06-27 23:30:22,035 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' ),' 2024-06-27 23:30:22,036 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' model=operation_model,' 2024-06-27 23:30:22,036 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' params=request_dict,' 2024-06-27 23:30:22,036 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' request_signer=self._request_signer,' 2024-06-27 23:30:22,036 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' context=request_context,' 2024-06-27 23:30:22,036 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' )' 2024-06-27 23:30:22,037 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' ' 2024-06-27 23:30:22,037 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' if event_response is not None:' 2024-06-27 23:30:22,037 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' http, parsed_response = event_response' 2024-06-27 23:30:22,037 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' else:' 2024-06-27 23:30:22,037 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' apply_request_checksum(request_dict)' 2024-06-27 23:30:22,037 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' http, parsed_response = self._make_request(' 2024-06-27 23:30:22,038 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' operation_model, request_dict, request_context' 2024-06-27 23:30:22,038 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' )' 2024-06-27 23:30:22,038 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' ' 2024-06-27 23:30:22,038 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' self.meta.events.emit(' 2024-06-27 23:30:22,038 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b" 'after-call.{service_id}.{operation_name}'.format(" 2024-06-27 23:30:22,039 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' service_id=service_id, operation_name=operation_name' 2024-06-27 23:30:22,039 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' ),' 2024-06-27 23:30:22,039 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' http_response=http,' 2024-06-27 23:30:22,039 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' ' 2024-06-27 23:30:22,040 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' parsed=parsed_response,' 2024-06-27 23:30:22,040 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' model=operation_model,' 2024-06-27 23:30:22,040 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' context=request_context,' 2024-06-27 23:30:22,041 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' )' 2024-06-27 23:30:22,041 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' ' 2024-06-27 23:30:22,041 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' if http.status_code >= 300:' 2024-06-27 23:30:22,041 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' error_code = parsed_response.get("Error", {}).get("Code")' 2024-06-27 23:30:22,041 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b' error_class = self.exceptions.from_code(error_code)' 2024-06-27 23:30:22,042 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'> raise error_class(parsed_response, operation_name)' 2024-06-27 23:30:22,042 (cephci.test_s3) [DEBUG] - cephci.ceph.py:1533 - b'E botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListBuckets operation: None' Version-Release number of selected component (if applicable): ceph version 18.2.1-198.el9cp How reproducible: always Steps to Reproduce: 1.deploy rhcs cluster 7.1 with rgw ssl 2.clone and configure s3tests repo 3.execute s3tests Actual results: sts tests failed with access denied for list buckets of alt user in teardown phase. this issue is seen with rgw ssl. Expected results: expected the tests pass without failures Additional info: fail log in rgw ssl suite: http://magna002.ceph.redhat.com/cephci-jenkins/hsm/s3tests_sse_s3_ssl_fix/cephci-run-UEOUDA/execute_s3tests_0.log rgw logs at debug level 20: http://magna002.ceph.redhat.com/cephci-jenkins/hsm/s3tests_sse_s3_ssl_fix/ceph-client.rgw.rgw.ssl.ceph-hsm-s3tests-k1rpe8-node7.wsjcww.log