Bug 2147342
| Summary: | RHOSP certification tests: cinderlib RecursionError: maximum recursion depth exceeded | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | adeiter |
| Component: | python-cinderlib | Assignee: | Cinder Bugs List <cinder-bugs> |
| Status: | NEW --- | QA Contact: | Evelina Shames <eshames> |
| Severity: | high | Docs Contact: | |
| Priority: | medium | ||
| Version: | 16.2 (Train) | CC: | ccopello, eharney, geguileo |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The cinderlib enablement/certification is optional in the OpenStack Cinder certification process, so it is not a blocker. The purpose of the opt-in cinderlib tests was to certify for usage in Ember-CSI (https://ember-csi.io/) and potentially in RHV in the future, so I would recommend you to opt-out of that part of the certification since none of those 2 usage paths are currently productized, so you won't be losing anything. As for the bug, there seems to exist 3 issues in the eventlet project related to this recursion issue (from newer to older): - https://github.com/eventlet/eventlet/issues/677 - https://github.com/eventlet/eventlet/issues/618 - https://github.com/eventlet/eventlet/issues/371 I would normally say that the reason why you experience this issue in OSP 16.2 and not in upstream master is that back in OSP16.2 cinderlib was more opinionated and always used eventlet, but that changed and its no longer the case in master. What doesn't add up is the driver failing in cinderlib usage but not when running in the Cinder service. |
Description of problem: RHOSP 16.2 certification tests failed on cinderlib RecursionError: maximum recursion depth exceeded Version-Release number of selected component (if applicable): * Red Hat Enterprise Linux release 8.4 (Ootpa) * RHOSP 16.2 * python3-cinderlib-1.0.1-2.20220110231458.199ebd4.el8ost.noarch * python3-cinder-15.6.1-2.20220112174913.c093eda.el8ost.noarch * openstack-cinder-15.6.1-2.20220112174913.c093eda.el8ost.noarch * python3-cinderclient-5.0.2-2.20220107174846.7e9e31c.el8ost.noarch How reproducible: Run RHOSP certification tests Steps to Reproduce: % python3 -m unittest2 discover -v cinderlib.tests.functional Actual results: test_list_supported_drivers (cinderlib.tests.functional.test_basic.BaseFunctTestCase) ... ok ====================================================================== ERROR: setUpClass (cinderlib.tests.functional.test_basic.BackendFunctBasic) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/cinderlib/tests/functional/base_tests.py", line 112, in setUpClass config['backends']] File "/usr/lib/python3.6/site-packages/cinderlib/tests/functional/base_tests.py", line 111, in <listcomp> cls.backends = [cinderlib.Backend(**cfg) for cfg in File "/usr/lib/python3.6/site-packages/cinderlib/cinderlib.py", line 87, in __init__ self.driver.do_setup(objects.CONTEXT) File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/infinidat.py", line 172, in do_setup self.management_address, auth, use_ssl=use_ssl) File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/infinidat.py", line 157, in _setup_and_get_system_object system.login() File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/infinibox.py", line 294, in login if self.compat.has_auth_sessions(): File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/compatibility.py", line 129, in has_auth_sessions return self._has_feature('api_auth_sessions') or self._has_feature('api/auth_sessions') File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/compatibility.py", line 91, in _has_feature return self._get_feature_version(feature_key, NOTHING) is not NOTHING File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/compatibility.py", line 84, in _get_feature_version self._init_features() File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/compatibility.py", line 72, in _init_features resp = self.system.api.get("_features", assert_success=False) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 33, in returned return self.request(http_method, path=path, **kwargs) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 511, in request returned = self._request(http_method, path, **kwargs) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 327, in _request self.system.check_version() File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/infinibox.py", line 116, in check_version if not self.compat.can_run_on_system(): File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/compatibility.py", line 37, in can_run_on_system version_string = self.system.get_version().split('-', 1)[0] File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/infinibox.py", line 232, in get_version return self.get_system_info('version') File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/infinibox.py", line 197, in get_system_info return self.components.system_component.get_field(field_name, **kwargs) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/system_object.py", line 175, in get_field return self.get_fields([field_name], **kwargs)[field_name] File "/usr/local/lib/python3.6/site-packages/infinisdk/core/system_object.py", line 211, in get_fields response = self.system.api.get(query) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 33, in returned return self.request(http_method, path=path, **kwargs) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 511, in request returned = self._request(http_method, path, **kwargs) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 340, in _request self.system.compat.initialize() File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/compatibility.py", line 34, in initialize self._init_features() File "/usr/local/lib/python3.6/site-packages/infinisdk/infinibox/compatibility.py", line 72, in _init_features resp = self.system.api.get("_features", assert_success=False) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 33, in returned return self.request(http_method, path=path, **kwargs) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 511, in request returned = self._request(http_method, path, **kwargs) File "/usr/local/lib/python3.6/site-packages/infinisdk/core/api/api.py", line 396, in _request response = self._session.send(prepared, **kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen chunked=chunked) File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 344, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 843, in _validate_conn conn.connect() File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 337, in connect cert_reqs=resolve_cert_reqs(self.cert_reqs), File "/usr/local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 281, in create_urllib3_context context.options |= options File "/usr/lib64/python3.6/ssl.py", line 423, in options super(SSLContext, SSLContext).options.__set__(self, value) File "/usr/lib64/python3.6/ssl.py", line 423, in options super(SSLContext, SSLContext).options.__set__(self, value) File "/usr/lib64/python3.6/ssl.py", line 423, in options super(SSLContext, SSLContext).options.__set__(self, value) File "/usr/lib64/python3.6/ssl.py", line 423, in options super(SSLContext, SSLContext).options.__set__(self, value) File "/usr/lib64/python3.6/ssl.py", line 423, in options super(SSLContext, SSLContext).options.__set__(self, value) File "/usr/lib64/python3.6/ssl.py", line 423, in options super(SSLContext, SSLContext).options.__set__(self, value) File "/usr/lib64/python3.6/ssl.py", line 423, in options super(SSLContext, SSLContext).options.__set__(self, value) File "/usr/lib64/python3.6/ssl.py", line 423, in options ... RecursionError: maximum recursion depth exceeded ---------------------------------------------------------------------- Ran 1 test in 1.153s FAILED (errors=1) Expected results: Success Root cause: RHOSP custom changes in requests/urllib3 Python libraries: * python3-requests-2.20 * python3-urllib3-1.24.2 and diff between upstream and RHOSP version of python3-requests-2.20 is: -# Attempt to enable urllib3's SNI support, if possible -try: - from urllib3.contrib import pyopenssl - pyopenssl.inject_into_urllib3() - - # Check cryptography version - from cryptography import __version__ as cryptography_version - _check_cryptography(cryptography_version) -except ImportError: - pass - Additional info: We found a workaround: # curl https://raw.githubusercontent.com/psf/requests/v2.20.0/requests/__init__.py > /usr/lib/python3.6/site-packages/requests/__init__.py References: The issue is around using of eventlet.monkey_patch: * https://review.openstack.org/#/c/154642/2/specs/eventlet-best-practices.rst@54 * https://github.com/eventlet/eventlet/pull/531