Bug 2134769

Description of problem (please be detailed as possible and provide log
snippests):
  Calling the `create-external-cluster-resources.py` script with --rgw-endpoint
  and --rgw-tls-cert-path parameters (first pointing to the RGW Endpoint,
  second to the path with certificate), fails with vague error:
    Execution Failed: unable to connect to endpoint: <IP>:443


Version of all relevant components (if applicable):
  Rook version (use rook version inside of a Rook Pod): v4.12.0-0.ffcae8e019e3e67f76c70c9badde72646034ec79
  Storage backend version (e.g. for ceph do ceph -v): ceph version 16.2.7-126.el8cp (fe0af61d104d48cb9d116cde6e593b5fc8c197e4) pacific (stable)


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
  I'm not able to use the script for RGW with SSL/TLS enabled.


Is there any workaround available to the best of your knowledge?
  No


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
  3


Can this issue reproducible?
  100%


Can this issue reproduce from the UI?
  N/A


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. prepare external Ceph cluster with SSL for RGW
2. run the `create-external-cluster-resources.py` script (with `--rgw-endpoint`
    and `--rgw-tls-cert-path` parameters)


Actual results:
  When is the create-external-cluster-resources.py script called with
  --rgw-endpoint and --rgw-tls-cert-path parameters (first pointing to the RGW
  Endpoint, second to the path with certificate, it fails with following vague
  error:

    Execution Failed: unable to connect to endpoint: <IP>:443

  The problem is, that the path from --rgw-tls-cert-path parameter is used in
  validate_rgw_endpoint_tls_cert, which return the content of the cert file.
  And then in the endpoint_dial method, the content of the cert file is passed
  to requests.head(...) as verify parameter:

    r = requests.head(ep, timeout=timeout, verify=cert)

  But it expect path to the cert, not the content of the cert.


Expected results:
  The create-external-cluster-resources.py script will properly handle the
  certificate provided via --rgw-tls-cert-path argument.

Additional info:
  There is also similar problem in the get_rgw_fsid method. In following code:

  ...   
          if self._arg_parser.rgw_tls_cert_path and not self._arg_parser.rgw_skip_tls:
              cert = self.validate_rgw_endpoint_tls_cert()
              verify = True
  ...
              r = requests.get(
                  request_url,
                  auth=S3Auth(access_key, secret_key, rgw_endpoint),
                  cert=cert,
                  verify=verify,
              )
  ...

  It uses both cert and verify arguments of the get method, but in wrong
  meaning[1]:

  >  cert – (optional) if String, path to ssl client cert file (.pem). If
  >     Tuple, (‘cert’, ‘key’) pair.
  >  verify – (optional) Either a boolean, in which case it controls whether we
  >     verify the server’s TLS certificate, or a string, in which case it must be
  >     a path to a CA bundle to use. Defaults to True.
  [1] https://requests.readthedocs.io/en/latest/api/

  This is required for https://issues.redhat.com/browse/RHSTOR-2537
  Upstream issue: https://github.com/rook/rook/issues/11060