RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2064343 - hmac and hashlib raising issues on FIPS env: _hashlib.UnsupportedDigestmodError: Unsupported digestmod functools.partial(<function __hash_new at 0x7f6c30c10700>, 'sha256')
Summary: hmac and hashlib raising issues on FIPS env: _hashlib.UnsupportedDigestmodErr...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: python3.9
Version: CentOS Stream
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Python Maintainers
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-03-15 15:49 UTC by Andre
Modified: 2025-01-06 19:01 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-04 12:22:35 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-115634 0 None None None 2022-03-15 15:53:11 UTC

Description Andre 2022-03-15 15:49:28 UTC
Description of problem:

~~~
digest = functools.partial(hashlib.new, 'sha256')
hmac.new("key".encode("utf8"), b'message', digest)
~~~

This code works fine on Centos9 without FIPS enabled, but when enabling FIPS we see the following error:

~~~
Traceback (most recent call last):
  File "/home/cloud-user/test_hmac.py", line 8, in <module>
    hmac.new("key".encode("utf8"), b'message', digest)
  File "/usr/lib64/python3.9/hmac.py", line 189, in new
    return HMAC(key, msg, digestmod)
  File "/usr/lib64/python3.9/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/usr/lib64/python3.9/hmac.py", line 69, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
_hashlib.UnsupportedDigestmodError: Unsupported digestmod functools.partial(<function __hash_new at 0x7f6c30c10700>, 'sha256')
~~~

Version-Release number of selected component (if applicable):
Python 3.9.10

How reproducible:
Always

Steps to Reproduce:
1. Run the script on Centos9 with fips enabled

Actual results:

~~~
Traceback (most recent call last):
  File "/home/cloud-user/test_hmac.py", line 8, in <module>
    hmac.new("key".encode("utf8"), b'message', digest)
  File "/usr/lib64/python3.9/hmac.py", line 189, in new
    return HMAC(key, msg, digestmod)
  File "/usr/lib64/python3.9/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/usr/lib64/python3.9/hmac.py", line 69, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
_hashlib.UnsupportedDigestmodError: Unsupported digestmod functools.partial(<function __hash_new at 0x7f6c30c10700>, 'sha256')
~~~

Expected results:

Same as without FIPS enabled

Additional info:
Script used to test: http://pastebin.test.redhat.com/1037225

Comment 1 Christian Heimes 2022-03-15 15:58:37 UTC
Does

   hmac.new(b'key', b'message', 'sha256')

work for you?

Comment 2 Christian Heimes 2022-03-15 16:05:06 UTC
You should either pass the name of a hash function or the hash constructor directly to hmac.new(). In both cases Python will use the OpenSSL EVP interface for MAC.

>>> import hmac, hashlib, functools

>>> hmac.new(b'key', b'message', 'sha256')
<hmac.HMAC object at 0x7fe9dbf24d60>
>>> hmac.new(b'key', b'message', hashlib.sha256)
<hmac.HMAC object at 0x7fe9dbe93950>

If you pass something else to HMAC, then it triggers a different code path that falls back to a pure Python implementation. This pure Python implementation is not FIPS compliant and therefore fails in FIPS mode:

>>> hmac.new(b'key', b'message', functools.partial(hashlib.new, 'sha256'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.9/hmac.py", line 189, in new
    return HMAC(key, msg, digestmod)
  File "/usr/lib64/python3.9/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/usr/lib64/python3.9/hmac.py", line 69, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
_hashlib.UnsupportedDigestmodError: Unsupported digestmod functools.partial(<function __hash_new at 0x7fe9dbea5c10>, 'sha256')

Comment 3 Dmitry Belyavskiy 2022-03-15 16:10:02 UTC
Could you please show the corresponding C code you are invoking?

Comment 4 Christian Heimes 2022-03-15 16:23:03 UTC
It's not an OpenSSL problem. It is a problem how the code attempts to construct an HMAC object with Python. The preferred and correct way is to either pass the name of a hash function or the hash object. The additional indirection with functools.partial() triggers a code path that is blocked in FIPS mode.

https://gitlab.com/redhat/centos-stream/rpms/python3.9/-/blob/c076ac88b9366e289bda88e3a9797c9384856963/00329-fips.patch#L264

Comment 5 Christian Heimes 2022-03-15 16:31:00 UTC
Original code is https://github.com/openstack/swift/blob/master/swift/common/middleware/tempurl.py#L753

I'm moving the ticket to Python team for visibility.

Comment 6 Petr Viktorin (pviktori) 2022-03-16 09:46:36 UTC
The issue is that in FIPS mode we need to ensure that only OpenSSL is used for the hashes. OpenSSL needs a string algorithm name, so if you pass in another object we need to "unwrap" it and get the algorithm name for it. (If that's not possible, upstream Pyhon will fall back to a slower Python implementation which calls that object, but that code path is disabled in FIPS mode.)
The "unwrapping" is only done for hash constructors. I guess we can allow partial(hashlib.new, ...) as well, but we're getting into whack-a-mole/diminishing-returns.

Could you use the `hash_algorithm` name directly, instead of the `digest` partial?


> Could you please show the corresponding C code you are invoking?

Start with the Python code which is roughly this: https://github.com/encukou/cpython/blob/fips_rhel9/Lib/hmac.py#L58
And the C: 
- _hmac_new: https://github.com/encukou/cpython/blob/fips_rhel9/Modules/_hashopenssl.c#L1493
- py_digest_by_digestmod: https://github.com/encukou/cpython/blob/fips_rhel9/Modules/_hashopenssl.c#L322

Comment 7 Petr Viktorin (pviktori) 2022-03-23 13:31:02 UTC
Could you use the `hash_algorithm` name directly, instead of the `digest` partial?

Comment 8 Tomas Orsava 2022-05-04 12:22:35 UTC
This has been without a response for a month and a half, please reopen if you want to discuss the issue further.

Comment 9 Red Hat Bugzilla 2023-09-15 01:52:45 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days

Comment 10 ben.blattberg 2025-01-06 19:01:28 UTC
I'm running into a similar problem and just wanted to note it here for others/myself when I forget:

I'm running Red Hat-patched Python; with pallets/Flask and pallets/itsdangerous, both updated to use a lazy loader rather than `hashlib.sha1` directly:

- https://github.com/pallets/flask/commit/db461112c70d5f2bf93c7a6ac27eeb665c232dd0
- https://github.com/pallets/itsdangerous/commit/7f4dcf83a07bb3d53f4e0e65ef1b43327b4cca90

With those two alterations (Python erroring, pallets using `_lazy_sha1`), I'm noticing failures in a FIPS environment. My current thought is to avoid the lazy loader and patch the objects to call the hash algorithm as their default, which I suppose is the whole point of having a lazy loading func in the first place.

But if there's another way to think about this issue, I'd be happy to hear it.


Note You need to log in before you can comment on or make changes to this bug.