RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 853890 - ccs_sync aborts in free (ccs_sync: double free or corruption)
Summary: ccs_sync aborts in free (ccs_sync: double free or corruption)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: ricci
Version: 6.3
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: rc
: ---
Assignee: Chris Feist
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks: 883504 960054
TreeView+ depends on / blocked
 
Reported: 2012-09-03 09:07 UTC by Frantisek Reznicek
Modified: 2015-11-16 01:14 UTC (History)
5 users (show)

Fixed In Version: ccs-0.16.2-64.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-11-21 21:52:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1673 0 normal SHIPPED_LIVE ricci bug fix and enhancement update 2013-11-20 21:52:49 UTC

Description Frantisek Reznicek 2012-09-03 09:07:42 UTC
Description of problem:

ccs_sync aborts in free (ccs_sync: double free or corruption).

When an user issues ccs_sync with multiple hosts like this:
ccs_sync -f /etc/cluster/cluster.conf 192.168.10.11 192.168.10.12 192.168.10.13

ccs_sync aborts due to double free or memory corruption in free():

[root@dhcp-lab-xxx ~]# ccs_sync -f /etc/cluster/cluster.conf 192.168.10.11 192.168.10.12 192.168.10.13
*** glibc detected *** ccs_sync: double free or corruption (top): 0x0000000001611b40 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75916)[0x7ff219624916]
/lib64/libc.so.6(+0x78443)[0x7ff219627443]
ccs_sync[0x402508]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7ff2195cdcdd]
ccs_sync[0x401d79]
======= Memory map: ========
00400000-00406000 r-xp 00000000 fd:01 536615                             /usr/bin/ccs_sync
00605000-00606000 rw-p 00005000 fd:01 536615                             /usr/bin/ccs_sync
015e9000-0162b000 rw-p 00000000 00:00 0                                  [heap]
7ff2185a8000-7ff2185be000 r-xp 00000000 fd:01 262149                     /lib64/libgcc_s-4.4.6-20120305.so.1
7ff2185be000-7ff2187bd000 ---p 00016000 fd:01 262149                     /lib64/libgcc_s-4.4.6-20120305.so.1
7ff2187bd000-7ff2187be000 rw-p 00015000 fd:01 262149                     /lib64/libgcc_s-4.4.6-20120305.so.1
7ff2187be000-7ff21881b000 r-xp 00000000 fd:01 262170                     /lib64/libfreebl3.so
7ff21881b000-7ff218a1a000 ---p 0005d000 fd:01 262170                     /lib64/libfreebl3.so
7ff218a1a000-7ff218a1b000 r--p 0005c000 fd:01 262170                     /lib64/libfreebl3.so
7ff218a1b000-7ff218a1c000 rw-p 0005d000 fd:01 262170                     /lib64/libfreebl3.so
7ff218a1c000-7ff218a20000 rw-p 00000000 00:00 0 
7ff218a20000-7ff218a46000 r-xp 00000000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218a46000-7ff218c46000 ---p 00026000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218c46000-7ff218c47000 r--p 00026000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218c47000-7ff218c48000 rw-p 00027000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218c48000-7ff218cd4000 r-xp 00000000 fd:01 527633                     /usr/lib64/libsqlite3.so.0.8.6
7ff218cd4000-7ff218ed3000 ---p 0008c000 fd:01 527633                     /usr/lib64/libsqlite3.so.0.8.6
7ff218ed3000-7ff218ed7000 rw-p 0008b000 fd:01 527633                     /usr/lib64/libsqlite3.so.0.8.6
7ff218ed7000-7ff218f13000 r-xp 00000000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff218f13000-7ff219113000 ---p 0003c000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff219113000-7ff219114000 r--p 0003c000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff219114000-7ff219115000 rw-p 0003d000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff219115000-7ff219198000 r-xp 00000000 fd:01 262185                     /lib64/libm-2.12.so
7ff219198000-7ff219397000 ---p 00083000 fd:01 262185                     /lib64/libm-2.12.so
7ff219397000-7ff219398000 r--p 00082000 fd:01 262185                     /lib64/libm-2.12.so
7ff219398000-7ff219399000 rw-p 00083000 fd:01 262185                     /lib64/libm-2.12.so
7ff219399000-7ff2193ae000 r-xp 00000000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2193ae000-7ff2195ad000 ---p 00015000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2195ad000-7ff2195ae000 r--p 00014000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2195ae000-7ff2195af000 rw-p 00015000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2195af000-7ff219738000 r-xp 00000000 fd:01 262177                     /lib64/libc-2.12.so
7ff219738000-7ff219938000 ---p 00189000 fd:01 262177                     /lib64/libc-2.12.so
7ff219938000-7ff21993c000 r--p 00189000 fd:01 262177                     /lib64/libc-2.12.so
7ff21993c000-7ff21993d000 rw-p 0018d000 fd:01 262177                     /lib64/libc-2.12.so
7ff21993d000-7ff219942000 rw-p 00000000 00:00 0 
7ff219942000-7ff219944000 r-xp 00000000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219944000-7ff219b44000 ---p 00002000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219b44000-7ff219b45000 r--p 00002000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219b45000-7ff219b46000 rw-p 00003000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219b46000-7ff219b5d000 r-xp 00000000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219b5d000-7ff219d5d000 ---p 00017000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219d5d000-7ff219d5e000 r--p 00017000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219d5e000-7ff219d5f000 rw-p 00018000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219d5f000-7ff219d63000 rw-p 00000000 00:00 0 
7ff219d63000-7ff219d9c000 r-xp 00000000 fd:01 262214                     /lib64/libnspr4.so
7ff219d9c000-7ff219f9b000 ---p 00039000 fd:01 262214                     /lib64/libnspr4.so
7ff219f9b000-7ff219f9c000 r--p 00038000 fd:01 262214                     /lib64/libnspr4.so
7ff219f9c000-7ff219f9e000 rw-p 00039000 fd:01 262214                     /lib64/libnspr4.so
7ff219f9e000-7ff219fa0000 rw-p 00000000 00:00 0 
7ff219fa0000-7ff219fa4000 r-xp 00000000 fd:01 266036                     /lib64/libplc4.so
7ff219fa4000-7ff21a1a3000 ---p 00004000 fd:01 266036                     /lib64/libplc4.so
7ff21a1a3000-7ff21a1a4000 r--p 00003000 fd:01 266036                     /lib64/libplc4.so
7ff21a1a4000-7ff21a1a5000 rw-p 00004000 fd:01 266036                     /lib64/libplc4.so
7ff21a1a5000-7ff21a1a8000 r-xp 00000000 fd:01 266037                     /lib64/libplds4.so
7ff21a1a8000-7ff21a3a7000 ---p 00003000 fd:01 266037                     /lib64/libplds4.so
7ff21a3a7000-7ff21a3a8000 r--p 00002000 fd:01 266037                     /lib64/libplds4.so
7ff21a3a8000-7ff21a3a9000 rw-p 00003000 fd:01 266037                     /lib64/libplds4.so
7ff21a3a9000-7ff21a3c9000 r-xp 00000000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a3c9000-7ff21a5c8000 ---p 00020000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a5c8000-7ff21a5ce000 r--p 0001f000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a5ce000-7ff21a5cf000 rw-p 00025000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a5cf000-7ff21a702000 r-xp 00000000 fd:01 528686                     /usr/lib64/libnss3.so
7ff21a702000-7ff21a902000 ---p 00133000 fd:01 528686                     /usr/lib64/libnss3.so
7ff21a902000-7ff21a907000 r--p 00133000 fd:01 528686                     /usr/lib64/libnss3.so
7ff21a907000-7ff21a909000 rw-p 00138000 fd:01 528686                     /usr/lib64/libnss3.soAborted (core dumped)


Core file analysis shows:
...
Core was generated by `ccs_sync -f /etc/cluster/cluster.conf 192.168.10.11 192.168.10.12 192.168.10.13'.
Program terminated with signal 6, Aborted.
#0  0x00007f265f2638a5 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.80.el6_3.5.x86_64 libgcc-4.4.6-4.el6.x86_64 libxml2-2.7.6-4.el6_2.4.x86_64 nspr-4.9.1-2.el6_3.x86_64 nss-3.13.5-1.el6_3.x86_64 nss-softokn-3.12.9-11.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 nss-util-3.13.5-1.el6_3.x86_64 sqlite-3.6.20-1.el6.x86_64 zlib-1.2.3-27.el6.x86_64
(gdb) info threads
* 1 Thread 0x7f2660f54700 (LWP 4162)  0x00007f265f2638a5 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007f265f2638a5 in raise () from /lib64/libc.so.6
#1  0x00007f265f265085 in abort () from /lib64/libc.so.6
#2  0x00007f265f2a0fe7 in __libc_message () from /lib64/libc.so.6
#3  0x00007f265f2a6916 in malloc_printerr () from /lib64/libc.so.6
#4  0x00007f265f2a9443 in _int_free () from /lib64/libc.so.6
#5  0x0000000000402508 in main (argc=<value optimized out>, argv=<value optimized out>) at ricci_conf.c:232
(gdb) quit


Version-Release number of selected component (if applicable):
# rpm -qa|grep -E 'clu|ricci|coro' | sort
cluster-glue-libs-1.0.5-6.el6.x86_64
clusterlib-3.0.12.1-32.el6_3.1.x86_64
corosync-1.4.1-7.el6.x86_64
corosynclib-1.4.1-7.el6.x86_64
modcluster-0.16.2-18.el6.x86_64
qpid-cpp-server-cluster-0.14-21.el6_3.x86_64
ricci-0.16.2-55.el6.x86_64
ricci-debuginfo-0.16.2-55.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. start 3 node cluster using below cluster.conf configuration
2. ccs_sync -f /etc/cluster/cluster.conf 192.168.10.11 192.168.10.12 192.168.10.13
3.
  
Actual results:
ccs_sync aborts.

Expected results:
ccs_sync should not abort.

Additional info:

[root@dhcp-lab-xxx ~]# ccs_sync -f /etc/cluster/cluster.conf 192.168.10.11 192.168.10.12 192.168.10.13
*** glibc detected *** ccs_sync: double free or corruption (top): 0x0000000001611b40 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75916)[0x7ff219624916]
/lib64/libc.so.6(+0x78443)[0x7ff219627443]
ccs_sync[0x402508]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7ff2195cdcdd]
ccs_sync[0x401d79]
======= Memory map: ========
00400000-00406000 r-xp 00000000 fd:01 536615                             /usr/bin/ccs_sync
00605000-00606000 rw-p 00005000 fd:01 536615                             /usr/bin/ccs_sync
015e9000-0162b000 rw-p 00000000 00:00 0                                  [heap]
7ff2185a8000-7ff2185be000 r-xp 00000000 fd:01 262149                     /lib64/libgcc_s-4.4.6-20120305.so.1
7ff2185be000-7ff2187bd000 ---p 00016000 fd:01 262149                     /lib64/libgcc_s-4.4.6-20120305.so.1
7ff2187bd000-7ff2187be000 rw-p 00015000 fd:01 262149                     /lib64/libgcc_s-4.4.6-20120305.so.1
7ff2187be000-7ff21881b000 r-xp 00000000 fd:01 262170                     /lib64/libfreebl3.so
7ff21881b000-7ff218a1a000 ---p 0005d000 fd:01 262170                     /lib64/libfreebl3.so
7ff218a1a000-7ff218a1b000 r--p 0005c000 fd:01 262170                     /lib64/libfreebl3.so
7ff218a1b000-7ff218a1c000 rw-p 0005d000 fd:01 262170                     /lib64/libfreebl3.so
7ff218a1c000-7ff218a20000 rw-p 00000000 00:00 0 
7ff218a20000-7ff218a46000 r-xp 00000000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218a46000-7ff218c46000 ---p 00026000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218c46000-7ff218c47000 r--p 00026000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218c47000-7ff218c48000 rw-p 00027000 fd:01 527890                     /usr/lib64/libnssdbm3.so
7ff218c48000-7ff218cd4000 r-xp 00000000 fd:01 527633                     /usr/lib64/libsqlite3.so.0.8.6
7ff218cd4000-7ff218ed3000 ---p 0008c000 fd:01 527633                     /usr/lib64/libsqlite3.so.0.8.6
7ff218ed3000-7ff218ed7000 rw-p 0008b000 fd:01 527633                     /usr/lib64/libsqlite3.so.0.8.6
7ff218ed7000-7ff218f13000 r-xp 00000000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff218f13000-7ff219113000 ---p 0003c000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff219113000-7ff219114000 r--p 0003c000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff219114000-7ff219115000 rw-p 0003d000 fd:01 527892                     /usr/lib64/libsoftokn3.so
7ff219115000-7ff219198000 r-xp 00000000 fd:01 262185                     /lib64/libm-2.12.so
7ff219198000-7ff219397000 ---p 00083000 fd:01 262185                     /lib64/libm-2.12.so
7ff219397000-7ff219398000 r--p 00082000 fd:01 262185                     /lib64/libm-2.12.so
7ff219398000-7ff219399000 rw-p 00083000 fd:01 262185                     /lib64/libm-2.12.so
7ff219399000-7ff2193ae000 r-xp 00000000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2193ae000-7ff2195ad000 ---p 00015000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2195ad000-7ff2195ae000 r--p 00014000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2195ae000-7ff2195af000 rw-p 00015000 fd:01 262228                     /lib64/libz.so.1.2.3
7ff2195af000-7ff219738000 r-xp 00000000 fd:01 262177                     /lib64/libc-2.12.so
7ff219738000-7ff219938000 ---p 00189000 fd:01 262177                     /lib64/libc-2.12.so
7ff219938000-7ff21993c000 r--p 00189000 fd:01 262177                     /lib64/libc-2.12.so
7ff21993c000-7ff21993d000 rw-p 0018d000 fd:01 262177                     /lib64/libc-2.12.so
7ff21993d000-7ff219942000 rw-p 00000000 00:00 0 
7ff219942000-7ff219944000 r-xp 00000000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219944000-7ff219b44000 ---p 00002000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219b44000-7ff219b45000 r--p 00002000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219b45000-7ff219b46000 rw-p 00003000 fd:01 262183                     /lib64/libdl-2.12.so
7ff219b46000-7ff219b5d000 r-xp 00000000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219b5d000-7ff219d5d000 ---p 00017000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219d5d000-7ff219d5e000 r--p 00017000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219d5e000-7ff219d5f000 rw-p 00018000 fd:01 262201                     /lib64/libpthread-2.12.so
7ff219d5f000-7ff219d63000 rw-p 00000000 00:00 0 
7ff219d63000-7ff219d9c000 r-xp 00000000 fd:01 262214                     /lib64/libnspr4.so
7ff219d9c000-7ff219f9b000 ---p 00039000 fd:01 262214                     /lib64/libnspr4.so
7ff219f9b000-7ff219f9c000 r--p 00038000 fd:01 262214                     /lib64/libnspr4.so
7ff219f9c000-7ff219f9e000 rw-p 00039000 fd:01 262214                     /lib64/libnspr4.so
7ff219f9e000-7ff219fa0000 rw-p 00000000 00:00 0 
7ff219fa0000-7ff219fa4000 r-xp 00000000 fd:01 266036                     /lib64/libplc4.so
7ff219fa4000-7ff21a1a3000 ---p 00004000 fd:01 266036                     /lib64/libplc4.so
7ff21a1a3000-7ff21a1a4000 r--p 00003000 fd:01 266036                     /lib64/libplc4.so
7ff21a1a4000-7ff21a1a5000 rw-p 00004000 fd:01 266036                     /lib64/libplc4.so
7ff21a1a5000-7ff21a1a8000 r-xp 00000000 fd:01 266037                     /lib64/libplds4.so
7ff21a1a8000-7ff21a3a7000 ---p 00003000 fd:01 266037                     /lib64/libplds4.so
7ff21a3a7000-7ff21a3a8000 r--p 00002000 fd:01 266037                     /lib64/libplds4.so
7ff21a3a8000-7ff21a3a9000 rw-p 00003000 fd:01 266037                     /lib64/libplds4.so
7ff21a3a9000-7ff21a3c9000 r-xp 00000000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a3c9000-7ff21a5c8000 ---p 00020000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a5c8000-7ff21a5ce000 r--p 0001f000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a5ce000-7ff21a5cf000 rw-p 00025000 fd:01 527268                     /usr/lib64/libnssutil3.so
7ff21a5cf000-7ff21a702000 r-xp 00000000 fd:01 528686                     /usr/lib64/libnss3.so
7ff21a702000-7ff21a902000 ---p 00133000 fd:01 528686                     /usr/lib64/libnss3.so
7ff21a902000-7ff21a907000 r--p 00133000 fd:01 528686                     /usr/lib64/libnss3.so
7ff21a907000-7ff21a909000 rw-p 00138000 fd:01 528686                     /usr/lib64/libnss3.soAborted (core dumped)
[root@dhcp-lab-xxx ~]# gdb `which ccs_sync` core.4162 
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-56.el6)
...
(gdb) info threads
* 1 Thread 0x7f2660f54700 (LWP 4162)  0x00007f265f2638a5 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007f265f2638a5 in raise () from /lib64/libc.so.6
#1  0x00007f265f265085 in abort () from /lib64/libc.so.6
#2  0x00007f265f2a0fe7 in __libc_message () from /lib64/libc.so.6
#3  0x00007f265f2a6916 in malloc_printerr () from /lib64/libc.so.6
#4  0x00007f265f2a9443 in _int_free () from /lib64/libc.so.6
#5  0x0000000000402508 in main (argc=<value optimized out>, argv=<value optimized out>) at ricci_conf.c:232
(gdb) quit
[root@dhcp-lab-106 ~]# ccs_sync
[root@dhcp-lab-106 ~]# clustat
Cluster Status for mycluster_el6vm @ Mon Sep  3 10:52:45 2012
Member Status: Quorate

 Member Name                                                     ID   Status
 ------ ----                                                     ---- ------
 192.168.10.11                                                       1 Online
 192.168.10.12                                                       2 Online, Local
 192.168.10.13                                                       3 Online

[root@dhcp-lab-106 ~]# cman_tool version
6.2.0 config 4
[root@dhcp-lab-106 ~]# grep cluster_version /etc/cluster/cluster.conf
[root@dhcp-lab-106 ~]# grep version /etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster config_version="5" name="mycluster_el6vm">


[root@dhcp-lab-106 ~]# cat /etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster config_version="5" name="mycluster_el6vm">
        <fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="30"/>
        <fence_xvmd/>
        <clusternodes>
                <clusternode name="192.168.10.11" nodeid="1" votes="1">
                        <fence>
                                <method name="1">
                                        <device domain="test" key_file="/etc/cluster/fence_xvm.key" name="fence_1"/>
                                </method>
                        </fence>
                </clusternode>
                <clusternode name="192.168.10.12" nodeid="2" votes="1">
                        <fence>
                                <method name="1">
                                        <device domain="test" key_file="/etc/cluster/fence_xvm.key" name="fence_2"/>
                                </method>
                        </fence>
                </clusternode>
                <clusternode name="192.168.10.13" nodeid="3" votes="1">
                        <fence>
                                <method name="1">
                                        <device domain="test" key_file="/etc/cluster/fence_xvm.key" name="fence_3"/>
                                </method>
                        </fence>
                </clusternode>
        </clusternodes>
        <cman port="1229">
                <multicast addr="225.0.0.12"/>
        </cman>
        <rm log_level="7">
                <failoverdomains>
                        <failoverdomain name="domain_qpidd_1" restricted="1">
                                <failoverdomainnode name="192.168.10.11" priority="1"/>
                        </failoverdomain>
                        <failoverdomain name="domain_qpidd_2" restricted="1">
                                <failoverdomainnode name="192.168.10.12" priority="1"/>
                        </failoverdomain>
                        <failoverdomain name="domain_qpidd_3" restricted="1">
                                <failoverdomainnode name="192.168.10.13" priority="1"/>
                        </failoverdomain>
                </failoverdomains>
                <resources>
                        <script file="/etc/init.d/qpidd" name="qpidd"/>
                </resources>
                <service domain="domain_qpidd_1" name="qpidd_1">
                        <script ref="qpidd"/>
                </service>
                <service domain="domain_qpidd_2" name="qpidd_2">
                        <script ref="qpidd"/>
                </service>
                <service domain="domain_qpidd_3" name="qpidd_3">
                        <script ref="qpidd"/>
                </service>
        </rm>
        <fencedevices>
                <fencedevice action="reboot" agent="fence_xvm" key_file="/etc/cluster/fence_xvm.key" name="fence_1"/>
                <fencedevice action="reboot" agent="fence_xvm" key_file="/etc/cluster/fence_xvm.key" name="fence_2"/>
                <fencedevice action="reboot" agent="fence_xvm" key_file="/etc/cluster/fence_xvm.key" name="fence_3"/>
        </fencedevices>
</cluster>

Comment 2 Frantisek Reznicek 2012-09-03 09:36:39 UTC
Workaround using cman_tool works well (cman_tool version -r)

Comment 3 Jan Pokorný [poki] 2012-09-03 17:39:03 UTC
Confirming, thanks for trying this corner case.

I proposed a (roughly tested) patch in (temporary) branch upstream:
http://git.fedorahosted.org/cgit/conga.git/log/?h=bz853890

Comment 6 Chris Feist 2012-09-17 22:53:16 UTC
I haven't had a lot of time to look at the code, but I did a quick test and wouldn't it be simpler just to pull the cmdline_nodes out of the if/else like this?


--- a/ricci/ccs_sync/ricci_conf.c
+++ b/ricci/ccs_sync/ricci_conf.c
@@ -68,6 +68,7 @@ int main(int argc, const char const * const *argv) {
        hash_t *dest_nodes;
        int num_nodes = 0;
        int increment_version = 0;
+       hash_t cmdline_nodes;
 
        ret = hash_init(&node_hash, 5, string_compare, hash_string, hash_dtor);
        if (ret == -1) {
@@ -181,7 +182,6 @@ int main(int argc, const char const * const *argv) {
                /* Send to all current member nodes */
                dest_nodes = &node_hash;
        } else {
-               hash_t cmdline_nodes;
                ret = hash_init(&cmdline_nodes, 5, string_compare,
                                hash_string, hash_dtor);
                if (ret == -1) {


That seems to work and only moves one line of code.  jpokorny do you see an issue with this code?

Comment 7 Jan Pokorný [poki] 2012-09-18 15:19:39 UTC
[re comment 6]

It works, but duct taping (naïve solution) is probably what we want to
avoid as it may bite in long-term, isn't it?

These are things not covered by your patch:
- dead code ("if (num_nodes <= 0)" condition)
- memory leak (either if nodes used selectively or all-in-cluster.conf)
  - missing hash_destroy
- (this is a bit more subjective) bad condition for "Unable to find
  cluster nodes in ..." message

As you can see, my version is even shorter, even though I added comments
to support comprehension.

Purely my opinion, I would definitely be more happy with the patch
I proposed, however your version will (probably) do the job from
purely blackbox perspective, too.

Comment 8 Chris Feist 2012-10-02 18:49:29 UTC
Then lets go with my solution.  The patch moves one line of code so it's unlikely to affect any other currently expected behavior with ccs.  If we were planning on using ccs_sync in future releases it would probably be worth fixing it, but for now, just making a one line change that fixes the problem is the way to go.

Comment 15 Chris Feist 2013-08-08 21:39:16 UTC
Before fix:

[root@ask-02 ~]# rpm -q ricci
ricci-0.16.2-63.el6.x86_64
[root@ask-02 ~]# ccs -f bztest --createcluster bztest -i
[root@ask-02 ~]# ccs -f bztest --addnode n1
Node n1 added.
[root@ask-02 ~]# ccs -f bztest --addnode n2
Node n2 added.
[root@ask-02 ~]# ccs_sync -f bztest n1 n2
*** glibc detected *** ccs_sync: double free or corruption (top): 0x0000000001ef97a0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3914475366]
/lib64/libc.so.6[0x3914477e93]
ccs_sync[0x4024b8]
....
After fix:  (make sure n1/n2 are nodes that exist in dns but aren't running ricci)
[root@ask-02 t2]# rpm -q ricci
ricci-0.16.2-64.el6.x86_64
[root@ask-02 t2]# ccs -f bztest --createcluster bztest -i
[root@ask-02 t2]# ccs -f bztest --addnode n1
Node n1 added.
[root@ask-02 t2]# ccs -f bztest --addnode n2
Node n2 added.
[root@ask-02 t2]# ccs_sync -f bztest n1 n2
Failed to connect to n2: Connection refused.
Failed to connect to n1: Connection refused.

Comment 18 errata-xmlrpc 2013-11-21 21:52:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1673.html


Note You need to log in before you can comment on or make changes to this bug.