Bug 1740218

Summary: [fips] knet transport fails to initialize with OS in fips mode
Product: Red Hat Enterprise Linux 8 Reporter: Patrik Hagara <phagara>
Component: pcsAssignee: Tomas Jelinek <tojeline>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: high Docs Contact:
Priority: high    
Version: 8.1CC: aherr, cfeist, cluster-maint, fdinitto, idevat, jfriesse, mmazoure, omular, tojeline, toneata
Target Milestone: rcKeywords: ZStream
Target Release: 8.1Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pcs-0.10.2-4.el8 Doc Type: Bug Fix
Doc Text:
Cause: Pcs creates 384 bytes long corosync authkey. Consequence: Corosync does not start when FIPS mode is enabled. Fix: Make pcs create 256 bytes long corosync authkey. Result: Corosync starts even when FIPS mode is enabled.
Story Points: ---
Clone Of:
: 1746565 (view as bug list) Environment:
Last Closed: 2019-11-05 20:40:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1746565    
Attachments:
Description Flags
proposed fix none

Description Patrik Hagara 2019-08-12 13:38:03 UTC
Description of problem:

> Aug 12 14:01:00 virt-063 systemd[1]: Starting Corosync Cluster Engine...
> Aug 12 14:01:00 virt-063 corosync[24924]:  [MAIN  ] Corosync Cluster Engine 3.0.2 starting up
> Aug 12 14:01:00 virt-063 corosync[24924]:  [MAIN  ] Corosync built-in features: dbus systemd xmlconf vqsim nozzle snmp pie relro bindnow
> Aug 12 14:01:01 virt-063 corosync[24924]:  [TOTEM ] Initializing transport (Kronosnet).
> Aug 12 14:01:01 virt-063 corosync[24924]:  [TOTEM ] knet_handle_crypto failed: -2
> Aug 12 14:01:01 virt-063 corosync[24924]:  [KNET  ] common: crypto_nss.so has been loaded from /usr/lib64/kronosnet/crypto_nss.so
> Aug 12 14:01:01 virt-063 corosync[24924]:  [KNET  ] nsscrypto: Failure to import key into NSS (-8190): security library: received bad data.
> Aug 12 14:01:01 virt-063 corosync[24924]:  [KNET  ] nsscrypto: Secret key is probably too long. Try reduce it to 256 bytes
> Aug 12 14:01:01 virt-063 corosync[24924]:  [MAIN  ] Can't initialize TOTEM layer
> Aug 12 14:01:01 virt-063 corosync[24924]:  [MAIN  ] Corosync Cluster Engine exiting with status 15 at main.c:1531.
> Aug 12 14:01:01 virt-063 systemd[1]: corosync.service: Main process exited, code=exited, status=15/n/a
> Aug 12 14:01:01 virt-063 systemd[1]: corosync.service: Failed with result 'exit-code'.
> Aug 12 14:01:01 virt-063 systemd[1]: Failed to start Corosync Cluster Engine.

Version-Release number of selected component (if applicable):
libknet1-1.10-1.el8.x86_64
corosync-3.0.2-3.el8.x86_64
pacemaker-2.0.2-2.el8.x86_64
pcs-0.10.2-3.el8.x86_64

How reproducible:
always

Steps to Reproduce:
1. enable fips mode on to-be cluster nodes
2. setup a cluster on the nodes using pcs
3. try to start the cluster

Actual results:
corosync fails to start due to knet crypto errors

Expected results:
cluster successfully starts in fips mode

Additional info:

Comment 1 Jan Friesse 2019-08-12 13:56:47 UTC
@Phagara:
this is nether kronosnet nor corosync bug. Error (and solution) is logged:

> Secret key is probably too long. Try reduce it to 256 bytes

and there is really nothing knet can do about it, because in fips mode it is simply impossible to import longer key. corosync-keygen generates 256 bytes key by default (exactly for this fips mode reason), but it looks like pcs is (for whatever reason) generating 384 bytes :(

Anyway, reassigning to pcs.

Comment 3 Tomas Jelinek 2019-08-13 12:12:45 UTC
Created attachment 1603296 [details]
proposed fix

Comment 4 Tomas Jelinek 2019-08-13 13:09:06 UTC
After fix:
[root@rh80-node1:~]# rpm -q pcs
pcs-0.10.2-4.el8.x86_64

[root@rh80-node1:~]# pcs cluster setup rhel80-knet rh80-node1 rh80-node2
No addresses specified for host 'rh80-node1', using 'rh80-node1'
No addresses specified for host 'rh80-node2', using 'rh80-node2'
Destroying cluster on hosts: 'rh80-node1', 'rh80-node2'...
rh80-node1: Successfully destroyed cluster
rh80-node2: Successfully destroyed cluster
Requesting remove 'pcsd settings' from 'rh80-node1', 'rh80-node2'
rh80-node1: successful removal of the file 'pcsd settings'
rh80-node2: successful removal of the file 'pcsd settings'
Sending 'corosync authkey', 'pacemaker authkey' to 'rh80-node1', 'rh80-node2'
rh80-node1: successful distribution of the file 'corosync authkey'
rh80-node1: successful distribution of the file 'pacemaker authkey'
rh80-node2: successful distribution of the file 'corosync authkey'
rh80-node2: successful distribution of the file 'pacemaker authkey'
Sending 'corosync.conf' to 'rh80-node1', 'rh80-node2'
rh80-node1: successful distribution of the file 'corosync.conf'
rh80-node2: successful distribution of the file 'corosync.conf'
Cluster has been successfully set up.

[root@rh80-node1:~]# ls -l /etc/corosync/authkey
-r--------. 1 root root 256 Aug 13 14:07 /etc/corosync/authkey

[root@rh80-node1:~]# fips-mode-setup --check
FIPS mode is enabled.

[root@rh80-node1:~]# pcs cluster start --all --wait
rh80-node2: Starting Cluster...
rh80-node1: Starting Cluster...
Waiting for node(s) to start...
rh80-node1: Started
rh80-node2: Started

[root@rh80-node1:~]# systemctl status corosync
● corosync.service - Corosync Cluster Engine
   Loaded: loaded (/usr/lib/systemd/system/corosync.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2019-08-13 14:08:09 CEST; 30s ago
{...snip...}

Comment 10 errata-xmlrpc 2019-11-05 20:40:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:3311