Description of problem: Deployment with SSL public endpoint fails while running task 'Clean up legacy Cinder keystone catalog' on undercloud. Version-Release number of selected component (if applicable): 16.2 How reproducible: Deploy with SSL public endpoints. The error is: ~~~ 2023-06-26 12:13:51.503466 | 52540014-f6ca-e10d-23f1-000000009e5e | FATAL | Clean up legacy Cinder keystone catalog entries | undercloud | error={"changed": false, "module_stderr": "Failed to discover available identity versions when contacting https://xx.yy.aa.bb:13000. Attempting to parse version from URL. Traceback (most recent call last): File \"/root/.local/lib/python3.6/site-packages/urllib3/connectionpool.py\", line 710, in urlopen chunked=chunked, File \"/root/.local/lib/python3.6/site-packages/urllib3/connectionpool.py\", line 386, in _make_request self._validate_conn(conn) File \"/root/.local/lib/python3.6/site-packages/urllib3/connectionpool.py\", line 1042, in _validate_conn conn.connect() File \"/root/.local/lib/python3.6/site-packages/urllib3/connection.py\", line 424, in connect tls_in_tls=tls_in_tls, File \"/root/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py\", line 453, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls) File \"/root/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py\", line 495, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock) File \"/usr/lib64/python3.6/ssl.py\", line 365, in wrap_socket _context=self, _session=session) File \"/usr/lib64/python3.6/ssl.py\", line 776, in __init__ self.do_handshake() File \"/usr/lib64/python3.6/ssl.py\", line 1036, in do_handshake self._sslobj.do_handshake() File \"/usr/lib64/python3.6/ssl.py\", line 648, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897) ~~~ We identified that the installation path '/root/.local/lib/python3.6/' is incorrect which should be '/usr/lib64/python3.6/*' Customer moved this local path and deployment went successful. Expected results: Customer claims to have not installed any additional python package. Why the task is looking for incorrect python variable although right path '/usr/lib64/python3.6/*' is already in place. Additional info: Ansible logs, tempate and sos of the undercloud in the supportshell. drwxrwxrwx. 3 yank yank 31 Jun 30 10:29 0090-templates.tar -rw-rw-rw-. 1 yank yank 8632200 Jun 28 03:06 0080-deploy-overcloud.log-2023-06-27 -rw-rw-rw-. 1 yank yank 7641745 Jun 26 12:16 0070-deploy-overcloud.log-2023-06-26-2 drwxrwxrwx. 2 yank yank 76 Jun 26 11:14 0060-ansible_log.tar drwxrwxrwx. 2 yank usbmon 70 Jun 26 10:44 sosreport-20230621-132905 drwxrwxrwx. 3 yank yank 28 Jun 21 14:13 0050-rhsupport.tar.gz drwxrwxrwx. 3 yank yank 74 Jun 21 13:52 0040-sosreport-cirs02ospooo01-03537999-2023-06-21-pxwnmau.tar.xz
This is not a bug but the expected behavior. If you install python using pip to the local user then you likely have the python binary in ~/.local/bin . Because the patch is included in the $PATH it would hide the global python but make the local python command with additional packages loaded. You can restore the ~/.local directory and run $ witch python3 to check which one of local python binary and system python binary is used. https://packaging.python.org/en/latest/tutorials/installing-packages/#installing-to-the-user-site We do not support installing additional packages by pip to user site, especially to some core users such as root, because it can break the usage of our python libraries. If the customer needs additional packages then it should not be installed in the user site but more limited scope such as virtualenv.
> Because the patch is included Because the *path* is included
Please check my previous comment. I don't think this is a bug or an even supported usage. The customer should not install additional packages to user site (or even global site) by pip.