RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1946623 - corosync crash with compression enabled
Summary: corosync crash with compression enabled
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: corosync
Version: 8.4
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: beta
: ---
Assignee: Jan Friesse
QA Contact: cluster-qe@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 1951049
TreeView+ depends on / blocked
 
Reported: 2021-04-06 14:20 UTC by Jan Friesse
Modified: 2021-11-09 21:01 UTC (History)
6 users (show)

Fixed In Version: corosync-3.1.0-4.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1951049 (view as bug list)
Environment:
Last Closed: 2021-11-09 17:49:23 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
Proposed patch (for 3.1.0) (2.08 KB, patch)
2021-04-06 14:26 UTC, Jan Friesse 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2021:4169 0 None None None 2021-11-09 17:49:33 UTC

Description Jan Friesse 2021-04-06 14:20:42 UTC 
Description of problem:
Corosync 3.1.0 (and 3.1.1) crashes when compression is enabled. Original upstream report https://github.com/corosync/corosync/issues/630

Version-Release number of selected component (if applicable):
3.1.0

How reproducible:
Memory corruption so 100% in valgrind, high probability otherwise

Steps to Reproduce:
1. Enable compression
2. Start corosync

Actual results:
Corosync crashes on startup

Expected results:
No crash

Additional info:
Fixed upstream 3.1.2. Corosync 3.0.x is not affected.
Comment 1 Jan Friesse 2021-04-06 14:26:17 UTC 
Created attachment 1769583 [details]
Proposed patch (for 3.1.0)
Comment 5 Dean Jansa 2021-04-12 14:00:47 UTC 
ON_QA bug without Verified:Tested should be in the MODIFIED state.
Comment 12 Patrik Hagara 2021-06-21 10:39:50 UTC 
before fix (corosync-3.1.0-3.el8)
=================================

> # pcs host auth virt-031 virt-032 virt-033 -u hacluster
> # pcs cluster setup rhel84 virt-031 virt-032 virt-033 transport knet compression level=6 model=zlib
> # pcs cluster corosync | grep compression
>     knet_compression_level: 6
>     knet_compression_model: zlib
> 
> 
> # pcs cluster start --all
> virt-033: Starting Cluster...
> virt-031: Starting Cluster...
> virt-032: Error connecting to virt-032 - (HTTP error: 400)
> Error: unable to start all nodes
> virt-032: Error connecting to virt-032 - (HTTP error: 400)
> 
> 
> [root@virt-032]# coredumpctl gdb 103646
>            PID: 103646 (corosync)
>            UID: 0 (root)
>            GID: 0 (root)
>         Signal: 11 (SEGV)
>      Timestamp: Wed 2021-04-21 12:52:42 CEST (2min 50s ago)
>   Command Line: /usr/sbin/corosync -f
>     Executable: /usr/sbin/corosync
>  Control Group: /system.slice/corosync.service
>           Unit: corosync.service
>          Slice: system.slice
>        Boot ID: ca297a1ad52449e3b8c4832ba7c11674
>     Machine ID: 5fcf2a5e7b1a4a93bac9dbfc69c3711e
>       Hostname: virt-032.cluster-qe.lab.eng.brq.redhat.com
>        Storage: /var/lib/systemd/coredump/core.corosync.0.ca297a1ad52449e3b8c4832ba7c11674.103646.1619002362000000.lz4
>        Message: Process 103646 (corosync) of user 0 dumped core.
>                 
>                 Stack trace of thread 103646:
>                 #0  0x00007fbf37653924 __pthread_mutex_lock (libpthread.so.0)
>                 #1  0x00007fbf37a7c53c get_global_wrlock (libknet.so.1)
>                 #2  0x00007fbf37a7695c knet_handle_compress (libknet.so.1)
>                 #3  0x00005568f4a58ba0 totemknet_configure_compression (corosync)
>                 #4  0x00005568f4a59dbd totemknet_initialize (corosync)
>                 #5  0x00005568f4a49c85 totemnet_initialize (corosync)
>                 #6  0x00005568f4a55421 totemsrp_initialize (corosync)
>                 #7  0x00005568f4a56a8b totempg_initialize (corosync)
>                 #8  0x00005568f4a25ae2 main (corosync)
>                 #9  0x00007fbf372a7493 __libc_start_main (libc.so.6)
>                 #10 0x00005568f4a262fe _start (corosync)

Result: corosync on random node crashes on startup when compression is enabled.


after fix (corosync-3.1.0-5.el8)
================================

> [root@virt-140 ~]# pcs host auth virt-140 virt-141 virt-143
> Username: hacluster
> Password: 
> virt-143: Authorized
> virt-141: Authorized
> virt-140: Authorized
> [root@virt-140 ~]# pcs cluster setup foo virt-140 virt-141 virt-143 transport knet compression level=6 model=zlib
> No addresses specified for host 'virt-140', using 'virt-140'
> No addresses specified for host 'virt-141', using 'virt-141'
> No addresses specified for host 'virt-143', using 'virt-143'
> Destroying cluster on hosts: 'virt-140', 'virt-141', 'virt-143'...
> virt-140: Successfully destroyed cluster
> virt-143: Successfully destroyed cluster
> virt-141: Successfully destroyed cluster
> Requesting remove 'pcsd settings' from 'virt-140', 'virt-141', 'virt-143'
> virt-140: successful removal of the file 'pcsd settings'
> virt-143: successful removal of the file 'pcsd settings'
> virt-141: successful removal of the file 'pcsd settings'
> Sending 'corosync authkey', 'pacemaker authkey' to 'virt-140', 'virt-141', 'virt-143'
> virt-140: successful distribution of the file 'corosync authkey'
> virt-140: successful distribution of the file 'pacemaker authkey'
> virt-143: successful distribution of the file 'corosync authkey'
> virt-143: successful distribution of the file 'pacemaker authkey'
> virt-141: successful distribution of the file 'corosync authkey'
> virt-141: successful distribution of the file 'pacemaker authkey'
> Sending 'corosync.conf' to 'virt-140', 'virt-141', 'virt-143'
> virt-140: successful distribution of the file 'corosync.conf'
> virt-143: successful distribution of the file 'corosync.conf'
> virt-141: successful distribution of the file 'corosync.conf'
> Cluster has been successfully set up.
> [root@virt-140 ~]# pcs cluster corosync | grep compression
>     knet_compression_level: 6
>     knet_compression_model: zlib
> [root@virt-140 ~]# pcs cluster start --all
> virt-141: Starting Cluster...
> virt-140: Starting Cluster...
> virt-143: Starting Cluster...
> [root@virt-140 ~]# pcs status
> Cluster name: foo
> 
> WARNINGS:
> No stonith devices and stonith-enabled is not false
> 
> Cluster Summary:
>   * Stack: corosync
>   * Current DC: virt-143 (version 2.1.0-2.el8-7c3f660707) - partition with quorum
>   * Last updated: Mon Jun 21 12:38:07 2021
>   * Last change:  Mon Jun 21 12:37:48 2021 by hacluster via crmd on virt-143
>   * 3 nodes configured
>   * 0 resource instances configured
> 
> Node List:
>   * Online: [ virt-140 virt-141 virt-143 ]
> 
> Full List of Resources:
>   * No resources
> 
> Daemon Status:
>   corosync: active/disabled
>   pacemaker: active/disabled
>   pcsd: active/enabled

Result: corosync startup succeeds with compression enabled.
Comment 14 errata-xmlrpc 2021-11-09 17:49:23 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (corosync bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4169
Note You need to log in before you can comment on or make changes to this bug.