Bug 1667061
Summary: | [RFE] provide commands for changing corosync configuration of an existing cluster | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 8 | Reporter: | Tomas Jelinek <tojeline> | ||||
Component: | pcs | Assignee: | Ondrej Mular <omular> | ||||
Status: | CLOSED ERRATA | QA Contact: | cluster-qe <cluster-qe> | ||||
Severity: | unspecified | Docs Contact: | Steven J. Levine <slevine> | ||||
Priority: | high | ||||||
Version: | 8.0 | CC: | ccaulfie, cfeist, cluster-maint, idevat, jfriesse, jpokorny, mlisik, mmazoure, nhostako, omular, slevine, tojeline | ||||
Target Milestone: | rc | Keywords: | FutureFeature, Triaged | ||||
Target Release: | 8.4 | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | pcs-0.10.7-3.el8 | Doc Type: | Enhancement | ||||
Doc Text: |
.New command to modify the Corosync configuration of an existing cluster
You can now modify the parameters of the `corosync.conf` file with the new `pcs cluster config update` command. You can use this command, for example, to increase the `totem` token to avoid fencing during temporary system unresponsiveness. For information on modifying the `corosync.conf` file, see link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_pcs-operation-configuring-and-managing-high-availability-clusters#proc_pcs-corosync-manage-pcs-operation[Modifying the corosync.conf file with the pcs command].
|
Story Points: | --- | ||||
Clone Of: | Environment: | ||||||
Last Closed: | 2021-05-18 15:12:05 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 1457314, 1856397 | ||||||
Attachments: |
|
Description
Tomas Jelinek
2019-01-17 11:00:21 UTC
Currently, only quorum configuration can be changed with 'pcs quorum' commands. Changing links configuration has been implemented in bz1667058 Honza, we would like to allow users to change as many configuration options as possible without stopping corosync. Therefore I need a list of corocync.conf option which can be changed in runtime. I found this list [1] but it is quite outdated (not covering knet at all). I'm aware of bz#1856397 but it only covers knet crypto options. I was unable to find any up-to-date source describing all the options. I'm primarily interested in options in totem section of corosync.conf as other sections (quorum, nodelist, totem.interface) are already covered in existing pcs commands. Do you have this documented somewhere or could you provide this information? Thanks. [1] https://github.com/corosync/corosync/wiki/Config-file-values @Onder, Hi. Sure. The link you've sent (https://github.com/corosync/corosync/wiki/Config-file-values) is actually almost correct. Definitive source of all available options is corosync.conf(5). According source, all options should be changeable during runtime with exception of: totem.version totem.threads (not used at all - to remove) totem.ip_version totem.rrp_mode (not used at all - to remove) totem.netmtu totem.interface.ringnumber (no used at all - to remove) totem.interface.bindnetaddr (not used when nodelist is) totem.interface.mcastaddr (not used when nodelist is) totem.interface.broadcast (not used when nodelist is) totem.interface.mcastport totem.interface.ttl totem.transport totem.cluster_name quorum.provider system.move_to_root_cgroup system.sched_rr system.priority system.qb_ipc_type system.state_dir @Chrissie: Aren't you aware about any other option which is not changeable in current master? Yes, you can't change totem.interface.<x>.knet_transport on the fly. You need to delete the link and re-create it. *** Bug 1173346 has been marked as a duplicate of this bug. *** Created attachment 1739694 [details]
proposed fix + tests
New command added:
* pcs cluster config update [transport <transport options>] [compression
<compression options>] [crypto <crypto options>] [totem <totem options>]
[--corosync_conf <path>]
Test:
pcs cluster config update transport knet_pmtud_interval=35 compression level=9 model=zlib crypto hash=sha512 totem token=10000 join=100
pcs cluster config update crypto cipher= hash= totem token=3000
Test: [root@r8-node-01 ~]# rpm -q pcs pcs-0.10.7-3.el8.x86_64 [root@r8-node-01 ~]# pcs cluster config Cluster Name: HACluster Transport: knet Nodes: r8-node-01 (nodeid: 1) r8-node-01 (link: 0) r8-node-02 (nodeid: 2) r8-node-02 (link: 0) Crypto Options: cipher: aes256 hash: sha256 [root@r8-node-01 ~]# pcs cluster config update transport knet_pmtud_interval=60 compression level=9 model=zlib threshold=100 crypto cipher=aes128 hash=sha512 model=nss totem token=10000 downcheck=2000 join=60 Sending updated corosync.conf to nodes... r8-node-01: Succeeded r8-node-02: Succeeded r8-node-01: Corosync configuration reloaded [root@r8-node-01 ~]# pcs cluster config Cluster Name: HACluster Transport: knet Nodes: r8-node-01 (nodeid: 1) r8-node-01 (link: 0) r8-node-02 (nodeid: 2) r8-node-02 (link: 0) Transport Options: knet_pmtud_interval: 60 Compression Options: level: 9 model: zlib threshold: 100 Crypto Options: cipher: aes128 hash: sha512 model: nss Totem Options: downcheck: 2000 join: 60 token: 10000 [root@r8-node-01 ~]# pcs cluster config update crypto hash= Error: If crypto option 'cipher' is enabled, crypto option 'hash' must be enabled as well Error: Errors have occurred, therefore pcs is unable to continue [root@r8-node-01 ~]# pcs cluster config update crypto hash= cipher= Sending updated corosync.conf to nodes... r8-node-02: Succeeded r8-node-01: Succeeded r8-node-01: Corosync configuration reloaded [root@r8-node-01 ~]# pcs cluster config Cluster Name: HACluster Transport: knet Nodes: r8-node-01 (nodeid: 1) r8-node-01 (link: 0) r8-node-02 (nodeid: 2) r8-node-02 (link: 0) Transport Options: knet_pmtud_interval: 60 Compression Options: level: 9 model: zlib threshold: 100 Crypto Options: model: nss Totem Options: downcheck: 2000 join: 60 token: 10000 Some options can't be changed during runtime: [root@r8-node-01 ~]# pcs cluster config update transport ip_version=ipv4 Checking corosync is not running on nodes... Error: r8-node-02: corosync is running Error: r8-node-01: corosync is running Error: Errors have occurred, therefore pcs is unable to continue AFTER: ====== [root@virt-025 ~]# rpm -q pcs corosync pcs-0.10.7-3.el8.x86_64 corosync-3.1.0-3.el8.x86_64 ## Changing ip_version - this option can't be changed in the running cluster [root@virt-025 ~]# pcs cluster config update transport ip_version=ipv4 Checking corosync is not running on nodes... Error: virt-025: corosync is running Error: virt-026: corosync is running Error: Errors have occurred, therefore pcs is unable to continue > OK: Error message is telling that the corosync is running [root@virt-025 ~]# pcs cluster stop --all virt-026: Stopping Cluster (pacemaker)... virt-025: Stopping Cluster (pacemaker)... virt-025: Stopping Cluster (corosync)... virt-026: Stopping Cluster (corosync)... [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Crypto Options: cipher: aes256 hash: sha256 [root@virt-025 ~]# pcs cluster config update transport ip_version=ipv4 Checking corosync is not running on nodes... virt-025: corosync is not running virt-026: corosync is not running Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded [root@virt-025 ~]# echo $? 0 > OK [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv4 Crypto Options: cipher: aes256 hash: sha256 > OK: The option is in the cluster config [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep ip_version ip_version: ipv4 > OK: The option is in the corosync.conf [root@virt-025 ~]# pcs cluster start --all virt-025: Starting Cluster... virt-026: Starting Cluster... [root@virt-025 ~]# corosync-cmapctl | grep ip_version totem.ip_version (str) = ipv4 > OK ## Functionality check [root@virt-025 ~]# cat /var/log/messages | grep 10.37.166.152, Jan 11 09:59:45 virt-025 corosync[592245]: [TOTEM ] Configured link number 0: local addr: 10.37.166.152, port=5405 > OK: Corosync is using ipv4 address [root@virt-025 ~]# pcs cluster stop --all virt-025: Stopping Cluster (pacemaker)... virt-026: Stopping Cluster (pacemaker)... virt-026: Stopping Cluster (corosync)... virt-025: Stopping Cluster (corosync)... [root@virt-025 ~]# pcs cluster config update transport ip_version=ipv6 Checking corosync is not running on nodes... virt-025: corosync is not running virt-026: corosync is not running Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded [root@virt-025 ~]# pcs cluster config | grep ip_version ip_version: ipv6 [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep ip_version ip_version: ipv6 [root@virt-025 ~]# pcs cluster start --all virt-025: Starting Cluster... virt-026: Starting Cluster... [root@virt-025 ~]# corosync-cmapctl | grep ip_version totem.ip_version (str) = ipv6 > OK [root@virt-025 ~]# cat /var/log/messages | grep 2620:52:0:25a4:1800:ff:fe00:19, Jan 11 10:27:25 virt-025 corosync[593763]: [TOTEM ] Configured link number 0: local addr: 2620:52:0:25a4:1800:ff:fe00:19, port=5405 > OK: Corosync is using ipv6 address ## Changing other transport options [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 Crypto Options: cipher: aes256 hash: sha256 [root@virt-025 ~]# pcs cluster config update transport knet_pmtud_interval=20 link_mode=rr Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# echo $? 0 [root@virt-025 ~]# pcs cluster config | grep "Transport Options" -A 3 Transport Options: ip_version: ipv6 knet_pmtud_interval: 20 link_mode: rr [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep "knet_pmtud_interval\|link_mode" knet_pmtud_interval: 20 link_mode: rr [root@virt-025 ~]# corosync-cmapctl | grep "knet_pmtud_interval\|link_mode" runtime.config.totem.knet_pmtud_interval (u32) = 20 totem.knet_pmtud_interval (u32) = 20 totem.link_mode (str) = rr > OK: The options were changed [root@virt-025 ~]# pcs cluster config update transport knet_pmtud_interval=25 link_mode=passive Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# pcs cluster config | grep "Transport Options" -A 3 Transport Options: ip_version: ipv6 knet_pmtud_interval: 25 link_mode: passive [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep "knet_pmtud_interval\|link_mode" knet_pmtud_interval: 25 link_mode: passive [root@virt-025 ~]# corosync-cmapctl | grep "knet_pmtud_interval\|link_mode" runtime.config.totem.knet_pmtud_interval (u32) = 25 totem.knet_pmtud_interval (u32) = 25 totem.link_mode (str) = passive > OK: The options with previous existing setting were changed ## Changing Compression options [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 knet_pmtud_interval: 25 link_mode: passive Crypto Options: cipher: aes256 hash: sha256 [root@virt-025 ~]# pcs cluster config update compression level=5 model=bzip2 threshold=1 Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# echo $? 0 > OK [root@virt-025 ~]# pcs cluster config | grep Compression -A 3 Compression Options: level: 5 model: bzip2 threshold: 1 [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep compression knet_compression_level: 5 knet_compression_model: bzip2 knet_compression_threshold: 1 [root@virt-025 ~]# corosync-cmapctl | grep "compression" runtime.config.totem.knet_compression_level (i32) = 5 runtime.config.totem.knet_compression_model (str) = bzip2 runtime.config.totem.knet_compression_threshold (u32) = 1 totem.knet_compression_level (i32) = 5 totem.knet_compression_model (str) = bzip2 totem.knet_compression_threshold (u32) = 1 > OK [root@virt-025 ~]# pcs cluster config update compression level=6 model=zlib threshold=150 Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# pcs cluster config | grep Compression -A 3 Compression Options: level: 6 model: zlib threshold: 150 [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep compression knet_compression_level: 6 knet_compression_model: zlib knet_compression_threshold: 150 [root@virt-025 ~]# corosync-cmapctl | grep "compression" runtime.config.totem.knet_compression_level (i32) = 6 runtime.config.totem.knet_compression_model (str) = zlib runtime.config.totem.knet_compression_threshold (u32) = 150 totem.knet_compression_level (i32) = 6 totem.knet_compression_model (str) = zlib totem.knet_compression_threshold (u32) = 150 > OK: The options with previous existing setting were changed ## Changing Crypto options [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 knet_pmtud_interval: 25 link_mode: passive Compression Options: level: 6 model: zlib threshold: 150 Crypto Options: cipher: aes256 hash: sha256 [root@virt-025 ~]# pcs cluster config update crypto cipher=aes192 hash=md5 model=openssl Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# echo $? 0 > OK [root@virt-025 ~]# pcs cluster config | grep Crypto -A 3 Crypto Options: cipher: aes192 hash: md5 model: openssl [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep crypto crypto_cipher: aes192 crypto_hash: md5 crypto_model: openssl [root@virt-025 ~]# corosync-cmapctl | grep crypto totem.crypto_cipher (str) = aes192 totem.crypto_hash (str) = md5 totem.crypto_model (str) = openssl > OK [root@virt-025 ~]# pcs cluster config update crypto cipher=aes128 hash=sha1 model=nss Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# pcs cluster config | grep Crypto -A 3 Crypto Options: cipher: aes128 hash: sha1 model: nss [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep crypto crypto_cipher: aes128 crypto_hash: sha1 crypto_model: nss [root@virt-025 ~]# corosync-cmapctl | grep crypto totem.crypto_cipher (str) = aes128 totem.crypto_hash (str) = sha1 totem.crypto_model (str) = nss > OK: The options with previous existing setting were changed ## Removing options from the config [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 knet_pmtud_interval: 25 link_mode: passive Compression Options: level: 6 model: zlib threshold: 150 Crypto Options: cipher: aes128 hash: sha1 model: nss [root@virt-025 ~]# pcs cluster config update crypto cipher= hash= model= transport knet_pmtud_interval= link_mode= compression level= model= threshold= Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 > OK ## Changing options from multiple categories at once [root@virt-025 ~]# pcs cluster config update transport knet_pmtud_interval=25 compression level=5 model=zlib Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# echo $? 0 [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 knet_pmtud_interval: 25 Compression Options: level: 5 model: zlib > OK [root@virt-025 ~]# pcs cluster config update transport knet_pmtud_interval= compression level= model= Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded ## Changing options in specified corosync_conf file [root@virt-025 ~]# cp /etc/corosync/corosync.conf my_conf.conf [root@virt-025 ~]# pcs cluster config update transport knet_pmtud_interval=25 compression level=5 model=zlib --corosync_conf my_conf.conf [root@virt-025 ~]# echo $? 0 [root@virt-025 ~]# cat my_conf.conf | grep 'knet_pmtud_interval\|level\|model' knet_pmtud_interval: 25 knet_compression_level: 5 knet_compression_model: zlib > OK: The options are changed in the specified file [root@virt-025 ~]# cat /etc/corosync/corosync.conf | grep 'knet_pmtud_interval\|level\|model' [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep 'knet_pmtud_interval\|level\|model' [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# corosync-cmapctl | grep 'knet_pmtud_interval\|level\|model' runtime.config.totem.knet_compression_level (i32) = 0 runtime.config.totem.knet_compression_model (str) = none runtime.config.totem.knet_pmtud_interval (u32) = 30 > OK: Changed options are only in the specified file ## Invalid inputs # Setting option with wrong value [root@virt-025 ~]# pcs cluster config update crypto hash=test Error: 'test' is not a valid hash value, use 'md5', 'none', 'sha1', 'sha256', 'sha384', 'sha512' Error: Errors have occurred, therefore pcs is unable to continue [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep hash [root@virt-025 ~]# echo $? 1 > OK [root@virt-025 ~]# pcs cluster config update transport knet_pmtud_interval=-5 Error: '-5' is not a valid knet_pmtud_interval value, use a non-negative integer Error: Errors have occurred, therefore pcs is unable to continue [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep knet_pmtud_interval [root@virt-025 ~]# echo $? 1 > OK # Setting option without fulfilled dependence on other option root@virt-025 ~]# pcs cluster config update crypto cipher=aes256 Error: If crypto option 'cipher' is enabled, crypto option 'hash' must be enabled as well Error: Errors have occurred, therefore pcs is unable to continue [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep cipher [root@virt-025 ~]# echo $? 1 > OK # Setting option under wrong category [root@virt-025 ~]# pcs cluster config update compression knet_pmtud_interval=10 Error: invalid compression option 'knet_pmtud_interval', allowed options are: 'level', 'model', 'threshold' Error: Errors have occurred, therefore pcs is unable to continue [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep knet_pmtud_interval [root@virt-025 ~]# echo $? 1 > OK [root@virt-025 ~]# pcs cluster config update transport knet_pmtud_interval=10 totem hash=sha1 Error: invalid totem option 'hash', allowed options are: 'consensus', 'downcheck', 'fail_recv_const', 'heartbeat_failures_allowed', 'hold', 'join', 'max_messages', 'max_network_delay', 'merge', 'miss_count_const', 'send_join', 'seqno_unchanged_const', 'token', 'token_coefficient', 'token_retransmit', 'token_retransmits_before_loss_const', 'window_size' Error: Errors have occurred, therefore pcs is unable to continue [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep knet_pmtud_interval [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep hash [root@virt-025 ~]# echo $? 1 > OK # Setting option without category [root@virt-025 ~]# pcs cluster config update knet_pmtud_interval=10 {...} \usage output {...} [root@virt-025 ~]# echo $? 1 [root@virt-025 ~]# pcs cluster config | grep knet_pmtud_interval [root@virt-025 ~]# echo $? 1 > OK # Inserting invalid option [root@virt-025 ~]# pcs cluster config update --test {...} \usage output {...} option --test not recognized [root@virt-025 ~]# echo $? 1 > OK # Setting non-existent corosync_conf file [root@virt-025 ~]# pcs cluster config update --corosync_conf temp.conf Error: Unable to read Corosync configuration 'temp.conf': No such file or directory: 'temp.conf' [root@virt-025 ~]# echo $? 1 > OK # Setting compression and crypto options with udp transport (tested on different cluster with udp transport set) [root@virt-022 ~]# rpm -q pcs pcs-0.10.7-3.el8.x86_64 [root@virt-022 ~]# pcs cluster config | grep Transport Transport: udp [root@virt-022 ~]# pcs cluster config update compression level=5 Error: The udp/udpu transport does not support 'compression' options, use 'knet' transport Error: Errors have occurred, therefore pcs is unable to continue [root@virt-022 ~]# echo $? 1 [root@virt-022 ~]# pcs cluster config | grep level [root@virt-022 ~]# echo $? 1 > OK [root@virt-022 ~]# pcs cluster config update crypto hash=sha256 Error: The udp/udpu transport does not support 'crypto' options, use 'knet' transport Error: Errors have occurred, therefore pcs is unable to continue [root@virt-022 ~]# echo $? 1 [root@virt-022 ~]# pcs cluster config | grep hash [root@virt-022 ~]# echo $? 1 > OK # Omitting options (not an invalid case) [root@virt-025 ~]# pcs cluster config update Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# echo $? 0 [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 [root@virt-025 ~]# pcs cluster config update totem transport Sending updated corosync.conf to nodes... virt-025: Succeeded virt-026: Succeeded virt-025: Corosync configuration reloaded [root@virt-025 ~]# echo $? 0 [root@virt-025 ~]# pcs cluster config Cluster Name: STSRHTS7522 Transport: knet Nodes: virt-025 (nodeid: 1) virt-025 (link: 0) virt-026 (nodeid: 2) virt-026 (link: 0) Transport Options: ip_version: ipv6 > Omitting a category or an option under a category is allowed as no setting will change Additional tests will follow. ## Changing totem options [root@virt-486 ~]# rpm -q pcs && rpm -q corosync pcs-0.10.7-3.el8.x86_64 corosync-3.1.0-3.el8.x86_64 # Default configuration, totem token is set to 3s [root@virt-486 ~]# pcs cluster config Cluster Name: STSRHTS14540 Transport: knet Nodes: virt-486 (nodeid: 1) virt-486 (link: 0) virt-487 (nodeid: 2) virt-487 (link: 0) Crypto Options: cipher: aes256 hash: sha256 [root@virt-486 ~]# pcs cluster corosync | grep -A6 totem totem { version: 2 cluster_name: STSRHTS14540 transport: knet crypto_cipher: aes256 crypto_hash: sha256 } [root@virt-486 ~]# corosync-cmapctl | grep totem.token runtime.config.totem.token (u32) = 3000 runtime.config.totem.token_retransmit (u32) = 714 runtime.config.totem.token_retransmits_before_loss_const (u32) = 4 runtime.config.totem.token_warning (u32) = 75 # Block corosync port on one of the nodes # Check the logs [root@virt-486 ~]# tail -f /var/log/messages ... Jan 11 12:25:28 virt-487 corosync[49822]: [TOTEM ] Token has not been received in 2250 ms Jan 11 12:25:29 virt-487 corosync[49822]: [TOTEM ] A processor failed, forming new configuration: token timed out (3000ms), waiting 3600ms for consensus. ... > Results in fencing after token times out # Unblock corosync # Raise totem token to 5s [root@virt-486 ~]# pcs cluster config update totem token=5000 Sending updated corosync.conf to nodes... virt-486: Succeeded virt-487: Succeeded virt-486: Corosync configuration reloaded [root@virt-486 ~]# echo $? 0 [root@virt-486 ~]# pcs cluster config | grep token token: 5000 [root@virt-486 ~]# pcs cluster corosync | grep -A7 totem totem { version: 2 cluster_name: STSRHTS14540 transport: knet crypto_cipher: aes256 crypto_hash: sha256 token: 5000 } [root@virt-486 ~]# corosync-cmapctl | grep totem.token runtime.config.totem.token (u32) = 5000 runtime.config.totem.token_retransmit (u32) = 1190 runtime.config.totem.token_retransmits_before_loss_const (u32) = 4 runtime.config.totem.token_warning (u32) = 75 totem.token (u32) = 5000 > OK, configuration and corosync.conf have changed # Block corosync port on one of the nodes # Check the logs [root@virt-486 ~]# tail -f /var/log/messages ... Jan 11 16:57:51 virt-486 corosync[49682]: [TOTEM ] Token has not been received in 3750 ms Jan 11 16:57:53 virt-486 corosync[49682]: [TOTEM ] A processor failed, forming new configuration: token timed out (5000ms), waiting 6000ms for consensus. ... > The change of token is apparent from the runtime configuration, logs and prolonged time for fencing to be triggered # Unblock corosync # Set token to the original value and try to update other parameters [root@virt-486 ~]# pcs cluster config update totem consensus=3000 downcheck=1200 fail_recv_const=3000 heartbeat_failures_allowed=1 hold=200 join=80 max_messages=15 max_network_delay=70 merge=150 miss_count_const=0 send_join=1 seqno_unchanged_const=20 token=3000 token_coefficient=800 token_retransmit=250 token_retransmits_before_loss_const=5 window_size=25 Sending updated corosync.conf to nodes... virt-486: Succeeded virt-487: Succeeded virt-486: Corosync configuration reloaded [root@virt-486 ~]# echo $? 0 [root@virt-486 ~]# pcs cluster config Cluster Name: STSRHTS14540 Transport: knet Nodes: virt-486 (nodeid: 1) virt-486 (link: 0) virt-487 (nodeid: 2) virt-487 (link: 0) Crypto Options: cipher: aes256 hash: sha256 Totem Options: consensus: 3000 downcheck: 1200 fail_recv_const: 3000 heartbeat_failures_allowed: 1 hold: 200 join: 80 max_messages: 15 max_network_delay: 70 merge: 150 miss_count_const: 0 send_join: 1 seqno_unchanged_const: 20 token: 3000 token_coefficient: 800 token_retransmit: 250 token_retransmits_before_loss_const: 5 window_size: 25 [root@virt-486 ~]# pcs cluster corosync | grep -A23 totem totem { version: 2 cluster_name: STSRHTS14540 transport: knet crypto_cipher: aes256 crypto_hash: sha256 token: 3000 consensus: 3000 downcheck: 1200 fail_recv_const: 3000 heartbeat_failures_allowed: 1 hold: 200 join: 80 max_messages: 15 max_network_delay: 70 merge: 150 miss_count_const: 0 send_join: 1 seqno_unchanged_const: 20 token_coefficient: 800 token_retransmit: 250 token_retransmits_before_loss_const: 5 window_size: 25 } # Check corosync.conf from another node [root@virt-486 ~]# pcs cluster corosync virt-487 | grep -A23 totem totem { version: 2 cluster_name: STSRHTS14540 transport: knet crypto_cipher: aes256 crypto_hash: sha256 token: 3000 consensus: 3000 downcheck: 1200 fail_recv_const: 3000 heartbeat_failures_allowed: 1 hold: 200 join: 80 max_messages: 15 max_network_delay: 70 merge: 150 miss_count_const: 0 send_join: 1 seqno_unchanged_const: 20 token_coefficient: 800 token_retransmit: 250 token_retransmits_before_loss_const: 5 window_size: 25 } [root@virt-486 ~]# corosync-cmapctl | grep totem config.totemconfig_reload_in_progress (u8) = 0 runtime.config.totem.block_unlisted_ips (u32) = 1 runtime.config.totem.consensus (u32) = 3000 runtime.config.totem.downcheck (u32) = 1200 runtime.config.totem.fail_recv_const (u32) = 3000 runtime.config.totem.heartbeat_failures_allowed (u32) = 1 runtime.config.totem.hold (u32) = 200 runtime.config.totem.interface.0.knet_ping_interval (u32) = 1250 runtime.config.totem.interface.0.knet_ping_timeout (u32) = 2500 runtime.config.totem.join (u32) = 80 runtime.config.totem.knet_compression_level (i32) = 0 runtime.config.totem.knet_compression_model (str) = none runtime.config.totem.knet_compression_threshold (u32) = 0 runtime.config.totem.knet_pmtud_interval (u32) = 30 runtime.config.totem.max_messages (u32) = 15 runtime.config.totem.max_network_delay (u32) = 70 runtime.config.totem.merge (u32) = 150 runtime.config.totem.miss_count_const (u32) = 5 runtime.config.totem.send_join (u32) = 1 runtime.config.totem.seqno_unchanged_const (u32) = 20 runtime.config.totem.token (u32) = 3000 runtime.config.totem.token_retransmit (u32) = 250 runtime.config.totem.token_retransmits_before_loss_const (u32) = 5 runtime.config.totem.token_warning (u32) = 75 runtime.config.totem.window_size (u32) = 25 totem.cluster_name (str) = STSRHTS14540 totem.consensus (u32) = 3000 totem.crypto_cipher (str) = aes256 totem.crypto_hash (str) = sha256 totem.downcheck (u32) = 1200 totem.fail_recv_const (u32) = 3000 totem.heartbeat_failures_allowed (u32) = 1 totem.hold (u32) = 200 totem.join (u32) = 80 totem.max_messages (u32) = 15 totem.max_network_delay (u32) = 70 totem.merge (u32) = 150 totem.miss_count_const (u32) = 0 totem.send_join (u32) = 1 totem.seqno_unchanged_const (u32) = 20 totem.token (u32) = 3000 totem.token_coefficient (u32) = 800 totem.token_retransmit (u32) = 250 totem.token_retransmits_before_loss_const (u32) = 5 totem.transport (str) = knet totem.version (u32) = 2 totem.window_size (u32) = 25 > OK # Revert the configuration to default values [root@virt-486 ~]# pcs cluster config update totem consensus= downcheck= fail_recv_const= heartbeat_failures_allowed= hold= join= max_messages= max_network_delay= merge= miss_count_const= send_join= seqno_unchanged_const= token= token_coefficient= token_retransmit= token_retransmits_before_loss_const= window_size= Sending updated corosync.conf to nodes... virt-486: Succeeded virt-487: Succeeded virt-486: Corosync configuration reloaded [root@virt-486 ~]# echo $? 0 [root@virt-486 ~]# pcs cluster config Cluster Name: STSRHTS14540 Transport: knet Nodes: virt-486 (nodeid: 1) virt-486 (link: 0) virt-487 (nodeid: 2) virt-487 (link: 0) Crypto Options: cipher: aes256 hash: sha256 [root@virt-486 ~]# pcs cluster corosync | grep -A6 totem totem { version: 2 cluster_name: STSRHTS14540 transport: knet crypto_cipher: aes256 crypto_hash: sha256 } [root@virt-486 ~]# corosync-cmapctl | grep totem config.totemconfig_reload_in_progress (u8) = 0 runtime.config.totem.block_unlisted_ips (u32) = 1 runtime.config.totem.consensus (u32) = 3600 runtime.config.totem.downcheck (u32) = 1000 runtime.config.totem.fail_recv_const (u32) = 2500 runtime.config.totem.heartbeat_failures_allowed (u32) = 0 runtime.config.totem.hold (u32) = 561 runtime.config.totem.interface.0.knet_ping_interval (u32) = 750 runtime.config.totem.interface.0.knet_ping_timeout (u32) = 1500 runtime.config.totem.join (u32) = 50 runtime.config.totem.knet_compression_level (i32) = 0 runtime.config.totem.knet_compression_model (str) = none runtime.config.totem.knet_compression_threshold (u32) = 0 runtime.config.totem.knet_pmtud_interval (u32) = 30 runtime.config.totem.max_messages (u32) = 17 runtime.config.totem.max_network_delay (u32) = 50 runtime.config.totem.merge (u32) = 200 runtime.config.totem.miss_count_const (u32) = 5 runtime.config.totem.send_join (u32) = 0 runtime.config.totem.seqno_unchanged_const (u32) = 30 runtime.config.totem.token (u32) = 3000 runtime.config.totem.token_retransmit (u32) = 714 runtime.config.totem.token_retransmits_before_loss_const (u32) = 4 runtime.config.totem.token_warning (u32) = 75 runtime.config.totem.window_size (u32) = 50 totem.cluster_name (str) = STSRHTS14540 totem.crypto_cipher (str) = aes256 totem.crypto_hash (str) = sha256 totem.transport (str) = knet totem.version (u32) = 2 > OK # Update the cluster configuration in a given file [root@virt-486 ~]# cp /etc/corosync/corosync.conf /tmp/corosync.conf [root@virt-486 ~]# pcs cluster config update totem consensus=3000 downcheck=1200 fail_recv_const=3000 heartbeat_failures_allowed=1 hold=200 join=80 max_messages=15 max_network_delay=70 merge=150 miss_count_const=0 send_join=1 seqno_unchanged_const=20 token=3000 token_coefficient=800 token_retransmit=250 token_retransmits_before_loss_const=5 window_size=25 --corosync_conf /tmp/corosync.conf [root@virt-486 ~]# echo $? 0 [root@virt-486 ~]# grep -A23 totem /tmp/corosync.conf totem { version: 2 cluster_name: STSRHTS14540 transport: knet crypto_cipher: aes256 crypto_hash: sha256 token: 3000 consensus: 3000 downcheck: 1200 fail_recv_const: 3000 heartbeat_failures_allowed: 1 hold: 200 join: 80 max_messages: 15 max_network_delay: 70 merge: 150 miss_count_const: 0 send_join: 1 seqno_unchanged_const: 20 token_coefficient: 800 token_retransmit: 250 token_retransmits_before_loss_const: 5 window_size: 25 } [root@virt-486 ~]# pcs cluster corosync | grep -A6 totem totem { version: 2 cluster_name: STSRHTS14540 transport: knet crypto_cipher: aes256 crypto_hash: sha256 } > OK, configuration is updated only in the specified file # Some of the totem options cannot be changed in an existing cluster [root@virt-486 ~]# pcs cluster config update totem cluster_name=test_name Error: invalid totem option 'cluster_name', allowed options are: 'consensus', 'downcheck', 'fail_recv_const', 'heartbeat_failures_allowed', 'hold', 'join', 'max_messages', 'max_network_delay', 'merge', 'miss_count_const', 'send_join', 'seqno_unchanged_const', 'token', 'token_coefficient', 'token_retransmit', 'token_retransmits_before_loss_const', 'window_size' Error: Errors have occurred, therefore pcs is unable to continue [root@virt-486 ~]# echo $? 1 > OK Marking verified in pcs-0.10.7-3.el8. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (pcs bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:1737 |