Bug 2326223 - conda fails on RHEL8 system in FIPS mode.
Summary: conda fails on RHEL8 system in FIPS mode.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: conda
Version: epel8
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Orion Poplawski
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-11-14 12:37 UTC by Brad Viviano
Modified: 2024-11-26 03:30 UTC (History)
2 users (show)

Fixed In Version: conda-4.10.3-2.el8
Clone Of:
Environment:
Last Closed: 2024-11-26 03:30:28 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Brad Viviano 2024-11-14 12:37:47 UTC
Description of problem:

The problem is actually in the python3-conda RPM, but there doesn't appear to be a separate component for that, so I am filing it under the conda component.

conda fails to work in FIPS environment if you use the non default Python because of use of hashlib.md5 in /usr/lib/python3.6/site-packages/conda/core/subdir_data.py

Per the Python docs, hashlib.md5 defaults to usedforsecurity=True and MD5 isn't a valid security cipher in FIPS mode.

Version-Release number of selected component (if applicable):

$ rpm -q conda python3-conda
conda-4.10.3-1.el8.noarch
python3-conda-4.10.3-1.el8.noarch

How reproducible:

Very

Steps to Reproduce:
1. Install conda onto a RHEL 8 system with FIPS enabled
2. Try and create a virtual environment with the non default Python version

Actual results:

[bviviano@atmos6 ~]$ ll /bin/python3.9 
-rwxr-xr-x. 1 root root 7776 Sep 26 21:02 /bin/python3.9
[bviviano@atmos6 ~]$ rpm -qf /bin/python3.9
python39-3.9.20-1.module+el8.10.0+22342+478c159e.x86_64
[bviviano@atmos6 ~]$ conda create --name myenv python=3.9
Collecting package metadata (current_repodata.json): failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/usr/lib/python3.6/site-packages/conda/exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "/usr/lib/python3.6/site-packages/conda/cli/main.py", line 84, in _main
        exit_code = do_call(args, p)
      File "/usr/lib/python3.6/site-packages/conda/cli/conda_argparse.py", line 83, in do_call
        return getattr(module, func_name)(args, parser)
      File "/usr/lib/python3.6/site-packages/conda/cli/main_create.py", line 41, in execute
        install(args, parser, 'create')
      File "/usr/lib/python3.6/site-packages/conda/cli/install.py", line 265, in install
        should_retry_solve=(_should_retry_unfrozen or repodata_fn != repodata_fns[-1]),
      File "/usr/lib/python3.6/site-packages/conda/core/solve.py", line 117, in solve_for_transaction
        should_retry_solve)
      File "/usr/lib/python3.6/site-packages/conda/core/solve.py", line 158, in solve_for_diff
        force_remove, should_retry_solve)
      File "/usr/lib/python3.6/site-packages/conda/core/solve.py", line 262, in solve_final_state
        ssc = self._collect_all_metadata(ssc)
      File "/usr/lib/python3.6/site-packages/conda/common/io.py", line 88, in decorated
        return f(*args, **kwds)
      File "/usr/lib/python3.6/site-packages/conda/core/solve.py", line 425, in _collect_all_metadata
        index, r = self._prepare(prepared_specs)
      File "/usr/lib/python3.6/site-packages/conda/core/solve.py", line 1021, in _prepare
        self.subdirs, prepared_specs, self._repodata_fn)
      File "/usr/lib/python3.6/site-packages/conda/core/index.py", line 289, in get_reduced_index
        repodata_fn=repodata_fn)
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 140, in query_all
        result = tuple(concat(executor.map(subdir_query, channel_urls)))
      File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
        yield fs.pop().result()
      File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
        return self.__get_result()
      File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
        raise self._exception
      File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 133, in <lambda>
        package_ref_or_match_spec))
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 145, in query
        self.load()
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 210, in load
        _internal_state = self._load()
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 319, in _load
        mtime = getmtime(self.cache_path_json)
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 203, in cache_path_json
        return self.cache_path_base + ('1' if context.use_only_tar_bz2 else '') + '.json'
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 195, in cache_path_base
        splitext(cache_fn_url(self.url_w_credentials, self.repodata_fn))[0])
      File "/usr/lib/python3.6/site-packages/conda/core/subdir_data.py", line 866, in cache_fn_url
        md5 = hashlib.md5(ensure_binary(url)).hexdigest()
    ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

`$ /usr/bin/conda create --name myenv python=3.9`

  environment variables:
                 CIO_TEST=<not set>
               CONDA_ROOT=/usr/share/conda
              CONDA_SHLVL=0
           CURL_CA_BUNDLE=<not set>
                  MANPATH=/usr/local/apps/lmod/lmod/share/man:
               MODULEPATH=/usr/local/apps/modulefiles/Compilers:/usr/local/apps/modulefiles/Appl
                          ications:/usr/local/apps/modulefiles/Core
                     PATH=/usr/bin:/usr/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sb
                          in:/opt/dell/srvadmin/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : None
            shell level : 0
       user config file : /home/bviviano/.condarc
 populated config files : /usr/share/conda/condarc.d/defaults.yaml
                          /home/bviviano/.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.6.8.final.0
       virtual packages : __linux=4.18.0=0
                          __glibc=2.28=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /usr  (read only)
      conda av data dir : /usr/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /var/cache/conda/pkgs
                          /home/bviviano/.conda/pkgs
       envs directories : /home/bviviano/.conda/envs
                          /usr/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.20.0 CPython/3.6.8 Linux/4.18.0-553.27.1.el8_10.x86_64 rhel/8.10 glibc/2.28
                UID:GID : 18228:50038
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.


Expected results:

[bviviano@atmos6 ~]$ conda create --name myenv python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/bviviano/.conda/envs/myenv

  added / updated specs:
    - python=3.9


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    _libgcc_mutex-0.1          |             main           3 KB
    _openmp_mutex-5.1          |            1_gnu          21 KB
    ca-certificates-2024.9.24  |       h06a4308_0         130 KB
    ld_impl_linux-64-2.40      |       h12ee557_0         710 KB
    libffi-3.4.4               |       h6a678d5_1         141 KB
    libgcc-ng-11.2.0           |       h1234567_1         5.3 MB
    libgomp-11.2.0             |       h1234567_1         474 KB
    libstdcxx-ng-11.2.0        |       h1234567_1         4.7 MB
    ncurses-6.4                |       h6a678d5_0         914 KB
    openssl-3.0.15             |       h5eee18b_0         5.2 MB
    pip-24.2                   |   py39h06a4308_0         2.2 MB
    python-3.9.20              |       he870216_1        25.1 MB
    readline-8.2               |       h5eee18b_0         357 KB
    setuptools-75.1.0          |   py39h06a4308_0         1.7 MB
    sqlite-3.45.3              |       h5eee18b_0         1.2 MB
    tk-8.6.14                  |       h39e8969_0         3.4 MB
    tzdata-2024b               |       h04d1e81_0         115 KB
    wheel-0.44.0               |   py39h06a4308_0         108 KB
    xz-5.4.6                   |       h5eee18b_1         643 KB
    zlib-1.2.13                |       h5eee18b_1         111 KB
    ------------------------------------------------------------
                                           Total:        52.5 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
  ca-certificates    pkgs/main/linux-64::ca-certificates-2024.9.24-h06a4308_0
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.40-h12ee557_0
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_1
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0
  openssl            pkgs/main/linux-64::openssl-3.0.15-h5eee18b_0
  pip                pkgs/main/linux-64::pip-24.2-py39h06a4308_0
  python             pkgs/main/linux-64::python-3.9.20-he870216_1
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0
  setuptools         pkgs/main/linux-64::setuptools-75.1.0-py39h06a4308_0
  sqlite             pkgs/main/linux-64::sqlite-3.45.3-h5eee18b_0
  tk                 pkgs/main/linux-64::tk-8.6.14-h39e8969_0
  tzdata             pkgs/main/noarch::tzdata-2024b-h04d1e81_0
  wheel              pkgs/main/linux-64::wheel-0.44.0-py39h06a4308_0
  xz                 pkgs/main/linux-64::xz-5.4.6-h5eee18b_1
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1


Proceed ([y]/n)? y


Downloading and Extracting Packages
ca-certificates-2024 | 130 KB    | ############################################################################# | 100% 
libstdcxx-ng-11.2.0  | 4.7 MB    | ############################################################################# | 100% 
ld_impl_linux-64-2.4 | 710 KB    | ############################################################################# | 100% 
_openmp_mutex-5.1    | 21 KB     | ############################################################################# | 100% 
tzdata-2024b         | 115 KB    | ############################################################################# | 100% 
libgomp-11.2.0       | 474 KB    | ############################################################################# | 100% 
libffi-3.4.4         | 141 KB    | ############################################################################# | 100% 
openssl-3.0.15       | 5.2 MB    | ############################################################################# | 100% 
xz-5.4.6             | 643 KB    | ############################################################################# | 100% 
libgcc-ng-11.2.0     | 5.3 MB    | ############################################################################# | 100% 
readline-8.2         | 357 KB    | ############################################################################# | 100% 
python-3.9.20        | 25.1 MB   | ############################################################################# | 100% 
pip-24.2             | 2.2 MB    | ############################################################################# | 100% 
zlib-1.2.13          | 111 KB    | ############################################################################# | 100% 
tk-8.6.14            | 3.4 MB    | ############################################################################# | 100% 
sqlite-3.45.3        | 1.2 MB    | ############################################################################# | 100% 
wheel-0.44.0         | 108 KB    | ############################################################################# | 100% 
_libgcc_mutex-0.1    | 3 KB      | ############################################################################# | 100% 
setuptools-75.1.0    | 1.7 MB    | ############################################################################# | 100% 
ncurses-6.4          | 914 KB    | ############################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate myenv
#
# To deactivate an active environment, use
#
#     $ conda deactivate

[bviviano@atmos6 ~]$ conda activate myenv
(myenv) [bviviano@atmos6 ~]$ python --version
Python 3.9.20
(myenv) [bviviano@atmos6 ~]$ 


Additional info:

I fixed this problem on my system by adding usedforsecurity=False to the hashlib.md5 function call in /usr/lib/python3.6/site-packages/conda/core/subdir_data.py:

    md5 = hashlib.md5(ensure_binary(url),usedforsecurity=False).hexdigest()

As noted in the python.org docs, usedforsecurity defaults to True, which will always fail on a FIPS enabled system.

Since the MD5 value is being used for integrity and not security in conda, I would request the maintainer make the same change for the next conda update.

Thanks!

Comment 1 Orion Poplawski 2024-11-17 00:38:52 UTC
So, I've applied this upstream patch - https://github.com/conda/conda/pull/11658, but this upstream issue is still open:
https://github.com/conda/conda/issues/7335

So there are likely still other issues with FIPS mode.

Comment 2 Fedora Update System 2024-11-17 03:17:44 UTC
FEDORA-EPEL-2024-cddd36e4d3 (conda-4.10.3-2.el8) has been submitted as an update to Fedora EPEL 8.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-cddd36e4d3

Comment 3 Fedora Update System 2024-11-18 00:50:20 UTC
FEDORA-EPEL-2024-cddd36e4d3 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-cddd36e4d3

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 4 Brad Viviano 2024-11-18 11:05:59 UTC
(In reply to Orion Poplawski from comment #1)
> So, I've applied this upstream patch -
> https://github.com/conda/conda/pull/11658, but this upstream issue is still
> open:
> https://github.com/conda/conda/issues/7335
> 
> So there are likely still other issues with FIPS mode.

Thanks for responding so quickly, there always seems to be other issues when running in FIPS more, regardless of the package :).

That said, the only location I see hashlib.md5 used in the RHEL8 EPEL conda package is what I reported:

[root@atmos4 site-packages]# pwd
/usr/lib/python3.6/site-packages
[root@atmos4 site-packages]# find conda* -type f -exec grep -H hashlib.md5 {} \;
conda/core/subdir_data.py:    md5 = hashlib.md5(ensure_binary(url),usedforsecurity=False).hexdigest()
[root@atmos4 site-packages]# 

The github issue linked claims there were more, but maybe that's in a newer version of conda then what is deployed in RHEL8 EPEL.

We've tested the code change on all our RHEL8 systems and haven't seen a problem and several of our users report they are able to use conda now to create and manage virtual environments, so I am hopeful there isn't any other code changes that need to be made, for this version of conda.

Comment 5 Fedora Update System 2024-11-26 03:30:28 UTC
FEDORA-EPEL-2024-cddd36e4d3 (conda-4.10.3-2.el8) has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.