1457314 – [RFE] Add commands for enabling and disabling cluster hardening in existing clusters

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1457314 - [RFE] Add commands for enabling and disabling cluster hardening in existing clusters

Summary: [RFE] Add commands for enabling and disabling cluster hardening in existing c...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 8
Classification:	Red Hat
Component:	pcs
Sub Component:
Version:	8.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	unspecified
Target Milestone:	rc
Target Release:	8.4
Assignee:	Ondrej Mular
QA Contact:	cluster-qe@redhat.com
Docs Contact:	Steven J. Levine
URL:
Whiteboard:
Depends On:	1667061 1856397
Blocks:
TreeView+	depends on / blocked

Reported:	2017-05-31 13:39 UTC by Tomas Jelinek
Modified:	2021-05-18 15:12 UTC (History)
CC List:	10 users (show)
Fixed In Version:	pcs-0.10.8-1.el8
Doc Type:	Enhancement
Doc Text:	.Enabling and disabling Corosync traffic encryption in an existing cluster Previously, you could configure Corosync traffic encryption only when creating a new cluster. With this update: * You can change the configuration of the Corosync crypto cipher and hash with the `pcs cluster config update` command. * You can change the Corosync `authkey` with the `pcs cluster authkey corosync` command.
Clone Of:
Environment:
Last Closed:	2021-05-18 15:12:05 UTC
Type:	Bug
Target Upstream Version:
Embargoed:
Dependent Products:
Flags:	pm-rhel: mirror+

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Red Hat Bugzilla	1165821	medium	CLOSED	pcs CLI/GUI should be capable of setting up a hardened cluster (for not entirely trusted environment)	2021-02-22 00:41:40 UTC
Red Hat Bugzilla	1173346	medium	CLOSED	Allow to modify arbitrary corosync.conf value	2021-06-10 10:49:13 UTC
Red Hat Bugzilla	1667061	high	CLOSED	[RFE] provide commands for changing corosync configuration of an existing cluster	2023-02-18 04:31:53 UTC

Internal Links: 1165821 1173346 1667061

Description Tomas Jelinek 2017-05-31 13:39:11 UTC

Since bz1165821 it is possible to enable cluster hardening when creating a new cluster. We want users to be able to enable and disable hardening without the need to destroy and recreate their clusters.

Comment 1 Tomas Jelinek 2017-05-31 13:57:42 UTC

When enabling encryption in an existing cluster, we want to remove all secauth, crypto_cipher and crypto_hash directives from corosnyc.conf. When these are not set, the encryption is enabled by default in corosync. Of course the authkey needs to be distributed as well.

Comment 2 Tomas Jelinek 2017-06-01 07:14:16 UTC

Also we should provide a command to synchronize the authkey in the cluster (pcs cluster sync?).

Comment 3 Jan Pokorný [poki] 2017-07-28 13:38:45 UTC

This should be just a fragment in the more generic change-what-you-want
(unless there's a technical obstavle) framework:
(a bit overdue) [bug 1173346].

Comment 4 Tomas Jelinek 2017-07-31 12:38:35 UTC

(In reply to Jan Pokorný from comment #3)
> This should be just a fragment in the more generic change-what-you-want
> (unless there's a technical obstavle) framework:
> (a bit overdue) [bug 1173346].

No, I don't think so.
When enabling corosync encryption, pcs needs to make sure that the same corosync authkey is present on all nodes. Meaning this is not just about editing the config.

Comment 6 Tomas Jelinek 2020-01-14 16:49:39 UTC

In RHEL 8, corosync authkey is distributed by pcs automatically to all cluster nodes. Therefore this just about editing the config.

Comment 14 Tomas Jelinek 2021-01-06 15:17:48 UTC

This is going to be implemented by bz1667061 and bz1856397

Comment 15 Tomas Jelinek 2021-01-28 16:11:35 UTC

Implemented by bz1667061 and bz1856397

Comment 16 Miroslav Lisik 2021-02-01 16:39:27 UTC

Test:

[root@r8-node-01 ~]# rpm -q pcs
pcs-0.10.8-1.el8.x86_64

### Disable cluster encryption:

[root@r8-node-01 ~]# pcs cluster config show
Cluster Name: HACluster
Transport: knet
Nodes:
  r8-node-01:
    Link 0 address: r8-node-01
    nodeid: 1
  r8-node-02:
    Link 0 address: r8-node-02
    nodeid: 2
Crypto Options:
  cipher: aes256
  hash: sha256

[root@r8-node-01 ~]# pcs cluster config update crypto cipher=none hash=none
Sending updated corosync.conf to nodes...
r8-node-02: Succeeded
r8-node-01: Succeeded
r8-node-01: Corosync configuration reloaded

[root@r8-node-01 ~]# journalctl -f -n 0 -u corosync.service
-- Logs begin at Mon 2021-02-01 11:31:16 CET. --
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [CFG   ] Config reload requested by node 1
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [TOTEM ] Configuring link 0
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [TOTEM ] Configured link number 0: local addr: 192.168.122.81, port=5405
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [TOTEM ] kronosnet crypto reconfigured on index 2: nss/none/none
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [KNET  ] nsscrypto: Digest does not match
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [KNET  ] nsscrypto: Digest does not match
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [KNET  ] nsscrypto: Digest does not match
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [KNET  ] nsscrypto: Digest does not match
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [KNET  ] pmtud: PMTUD link change for host: 2 link: 0 from 1397 to 1446
Feb 01 11:44:35 r8-node-01 corosync[6153]:   [KNET  ] pmtud: Global data MTU changed to: 1446

[root@r8-node-01 ~]# pcs cluster config
Cluster Name: HACluster
Transport: knet
Nodes:
  r8-node-01:
    Link 0 address: r8-node-01
    nodeid: 1
  r8-node-02:
    Link 0 address: r8-node-02
    nodeid: 2
Crypto Options:
  cipher: none
  hash: none

### Enable cluster encryption:

[root@r8-node-01 ~]# pcs cluster config update crypto cipher=aes128 hash=sha512
Sending updated corosync.conf to nodes...
r8-node-01: Succeeded
r8-node-02: Succeeded
r8-node-01: Corosync configuration reloaded

[root@r8-node-01 ~]# journalctl -f -n 0 -u corosync.service
-- Logs begin at Mon 2021-02-01 11:31:16 CET. --
Feb 01 11:46:14 r8-node-01 corosync[6153]:   [CFG   ] Config reload requested by node 1
Feb 01 11:46:14 r8-node-01 corosync[6153]:   [TOTEM ] Configuring link 0
Feb 01 11:46:14 r8-node-01 corosync[6153]:   [TOTEM ] Configured link number 0: local addr: 192.168.122.81, port=5405
Feb 01 11:46:14 r8-node-01 corosync[6153]:   [TOTEM ] kronosnet crypto reconfigured on index 1: nss/aes128/sha512
Feb 01 11:46:14 r8-node-01 corosync[6153]:   [KNET  ] pmtud: PMTUD link change for host: 2 link: 0 from 1446 to 1365
Feb 01 11:46:14 r8-node-01 corosync[6153]:   [KNET  ] pmtud: Global data MTU changed to: 1365

[root@r8-node-01 ~]# pcs cluster config
Cluster Name: HACluster
Transport: knet
Nodes:
  r8-node-01:
    Link 0 address: r8-node-01
    nodeid: 1
  r8-node-02:
    Link 0 address: r8-node-02
    nodeid: 2
Crypto Options:
  cipher: aes128
  hash: sha512


### Change corosync authkey:

[root@r8-node-01 ~]# for node in r8-node-0{1,2}; do ssh root@r8-node-01 "md5sum /etc/corosync/authkey"; done
c8b238ce4c511d3d7c3111814df7c117  /etc/corosync/authkey
c8b238ce4c511d3d7c3111814df7c117  /etc/corosync/authkey

[root@r8-node-01 ~]# pcs cluster authkey corosync
Sending 'corosync authkey' to 'r8-node-01', 'r8-node-02'
r8-node-01: successful distribution of the file 'corosync authkey'
r8-node-02: successful distribution of the file 'corosync authkey'
r8-node-01: Corosync configuration reloaded

[root@r8-node-01 ~]# journalctl -f -n 0 -u corosync.service
-- Logs begin at Mon 2021-02-01 11:31:16 CET. --
Feb 01 11:49:37 r8-node-01 corosync[6153]:   [CFG   ] Config reload requested by node 1
Feb 01 11:49:37 r8-node-01 corosync[6153]:   [TOTEM ] Configuring link 0
Feb 01 11:49:37 r8-node-01 corosync[6153]:   [TOTEM ] Configured link number 0: local addr: 192.168.122.81, port=5405
Feb 01 11:49:37 r8-node-01 corosync[6153]:   [TOTEM ] kronosnet crypto reconfigured on index 1: nss/aes128/sha512
Feb 01 11:49:37 r8-node-01 corosync[6153]:   [KNET  ] pmtud: Global data MTU changed to: 1365

[root@r8-node-01 ~]# for node in r8-node-0{1,2}; do ssh root@r8-node-01 "md5sum /etc/corosync/authkey"; done
c66ff61b202e1b0aba39153d7fa19728  /etc/corosync/authkey
c66ff61b202e1b0aba39153d7fa19728  /etc/corosync/authkey

Comment 27 errata-xmlrpc 2021-05-18 15:12:05 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pcs bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:1737

Note You need to log in before you can comment on or make changes to this bug.