Bug 1853302

Summary: Installation in FIPS mode fails on BareMetal IPI with error: "disabled for FIPS"
Product: OpenShift Container Platform Reporter: Raviv Bar-Tal <rbartal>
Component: Bare Metal Hardware ProvisioningAssignee: Dmitry Tantsur <dtantsur>
Bare Metal Hardware Provisioning sub component: ironic QA Contact: Ori Michaeli <omichael>
Status: CLOSED ERRATA Docs Contact:
Severity: low    
Priority: medium CC: beth.white, bfournie, rpittau, tsedovic
Version: 4.5Keywords: Triaged
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: ironic was using MD5 to build its hash ring. This hash function is not available in the FIPS 140-2 mode. Consequence: installation would fail because ironic would not be able to build a hash ring. Fix: the ironic image now defaults to SHA256 as a hash function for the hash ring.
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:11:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1869183    
Bug Blocks: 1804232    

Description Raviv Bar-Tal 2020-07-02 11:47:46 UTC
Description of problem:
Deployment of OCP-4.5 with fips on baremetal(vms simulating baremetal) fails:
openshift installer shows the below  error message:
time="2020-07-02T10:08:53Z" level=error msg="Error: Internal Server Error"
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error msg="  on ../../tmp/openshift-install-923937140/masters/main.tf line 1, in resource \"ironic_node_v1\" \"openshift-master-host\":"  
time="2020-07-02T10:08:53Z" level=error msg="   1: resource \"ironic_node_v1\" \"openshift-master-host\" {"                                                                 
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error msg="Error: Internal Server Error"
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error msg="  on ../../tmp/openshift-install-923937140/masters/main.tf line 1, in resource \"ironic_node_v1\" \"openshift-master-host\":"  
time="2020-07-02T10:08:53Z" level=error msg="   1: resource \"ironic_node_v1\" \"openshift-master-host\" {"                                                                 
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error msg="Error: Internal Server Error"
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error msg="  on ../../tmp/openshift-install-923937140/masters/main.tf line 1, in resource \"ironic_node_v1\" \"openshift-master-host\":"  
time="2020-07-02T10:08:53Z" level=error msg="   1: resource \"ironic_node_v1\" \"openshift-master-host\" {"                                                                 
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=error
time="2020-07-02T10:08:53Z" level=fatal msg="failed to fetch Cluster: failed to generate asset \"Cluster\": failed to create cluster: failed to apply Terraform: failed to complete the change"

Ironic-api on bootstrap shows the below error message:

2020-07-02 10:08:52.637 27 ERROR wsme.api [req-8ae567e1-29fc-4e1b-b3e4-14dde5921def - - - - -] Server-side error: "[digital envelope routines: EVP_DigestInit_ex] disabled for FIPS". Detail:
Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/wsmeext/pecan.py", line 85, in callfunction
    result = f(self, *args, **kwargs)

  File "/usr/lib/python3.6/site-packages/ironic/api/controllers/v1/node.py", line 2049, in post                                                                             
    topic = api.request.rpcapi.get_topic_for(node)

  File "/usr/lib/python3.6/site-packages/ironic/conductor/rpcapi.py", line 161, in get_topic_for                                                                            
    hostname = self.get_conductor_for(node)

  File "/usr/lib/python3.6/site-packages/ironic/conductor/rpcapi.py", line 144, in get_conductor_for                                                                        
    node.conductor_group)

  File "/usr/lib/python3.6/site-packages/ironic/common/hash_ring.py", line 84, in get_ring                                                                                  
    return self._get_ring(driver_name, conductor_group)

  File "/usr/lib/python3.6/site-packages/ironic/common/hash_ring.py", line 98, in _get_ring                                                                                 
    if not self.ring:

  File "/usr/lib/python3.6/site-packages/ironic/common/hash_ring.py", line 58, in ring                                                                                      
    rings = self._load_hash_rings()

  File "/usr/lib/python3.6/site-packages/ironic/common/hash_ring.py", line 72, in _load_hash_rings                                                                          
    hosts, partitions=2 ** CONF.hash_partition_exponent)

  File "/usr/lib/python3.6/site-packages/tooz/hashring.py", line 48, in __init__
    self.add_nodes(set(nodes))

  File "/usr/lib/python3.6/site-packages/tooz/hashring.py", line 73, in add_nodes
    key_hash = hashlib.md5(key)

ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
ESC[00m

Version-Release number of selected component (if applicable):


How reproducible:
set the fips option to true:
In QE Jenkins -> check FIPS_MODE checkbox
Manual installation -> set "fips: true " in install_config.yaml

Steps to Reproduce:
1. Set FIPS to true 
2. Run the installation 
3.

Actual results:
Installation fails

Expected results:


Additional info:

Comment 2 Tomas Sedovic 2020-08-13 11:27:26 UTC
All the pull requests have been merged. Setting to POST, but we need to tag the RPMs at which point we'll move this to MODIFIED.

Comment 3 Dmitry Tantsur 2020-08-24 07:26:34 UTC
Ready for testing.

Comment 4 Ori Michaeli 2020-08-27 06:59:25 UTC
Tested with 4.6.0-0.nightly-2020-08-26-064537:

Installation completed successfully and install-config was updated with fips: true

Comment 7 errata-xmlrpc 2020-10-27 16:11:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196