RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 617161 - config reload allows configuration to be downgraded
Summary: config reload allows configuration to be downgraded
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: cluster
Version: 6.0
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Fabio Massimo Di Nitto
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks: 619680 625560
TreeView+ depends on / blocked
 
Reported: 2010-07-22 11:15 UTC by Fabio Massimo Di Nitto
Modified: 2010-11-10 20:00 UTC (History)
7 users (show)

Fixed In Version: cluster-3.0.12-18.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 619680 (view as bug list)
Environment:
Last Closed: 2010-11-10 20:00:01 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
test cases (3.82 KB, text/plain)
2010-07-27 14:49 UTC, Fabio Massimo Di Nitto 		no flags Details
View All

Description Fabio Massimo Di Nitto 2010-07-22 11:15:12 UTC 
(this assume that the fix for #616222 to be in place or corosync will spin the CPU 100%)

start the cluster with cluster.conf at version 10

cman_tool version
6.2.0 config 10

edit cluster.conf to version 8

cman_tool version -r(any number < 8)

cman_tool version 
6.2.0 config 8

this should never be the case....
Comment 1 Lon Hohberger 2010-07-22 13:50:49 UTC 
If we tell people that use '-r0' or [proposed] configupdate, then I agree - it should not allow decrease.

If people use explicitly -r(X-2) (ex: restore from backup config by force), maybe it's not a big deal.

CMAN doesn't care that the config decreases; it only cares that the config is the same across the nodes.  A quorate cluster w/ version 10 will reject joins from a cluster member claiming version 12.
Comment 2 Lon Hohberger 2010-07-22 13:51:38 UTC 
OTOH, I do not know if other daemons rely on non-decreasing config_version ...  So, you're probably right.
Comment 3 Fabio Massimo Di Nitto 2010-07-22 17:24:48 UTC 
There are several problems involved allowing config downgrades, but that´s not the only issue.

First of all, the behavior is triggered by a bug in cman config reload code.

Second, we can´t assume it´s ok to downgrade. Other daemons and config interfaces might get a fit.

We always assumed config is incremental and we need to stick to that.
Comment 4 Fabio Massimo Di Nitto 2010-07-26 13:14:15 UTC 
http://git.fedorahosted.org/git/?p=cluster.git;a=commitdiff;h=ea01b37babe400a1f4545bd65e89d0ee6b35880e
Comment 5 Fabio Massimo Di Nitto 2010-07-27 08:50:34 UTC 
Note for QE: the test cases for those fixes are complex. I´ll need to prepare a proper document comparing before and after the fix.

Both Lon and I did test the fixes so I have built for Snap9. Stay tuned.
Comment 6 Fabio Massimo Di Nitto 2010-07-27 14:49:02 UTC 
Created attachment 434712 [details]
test cases

test cases to reproduce the issues with the old packages and how it works with the new ones.
Comment 10 Dean Jansa 2010-07-29 20:16:50 UTC 
Initial sanity check failed:

5 node cluster:

----- On all nodes, cman_tool version shows:

#  cman_tool version
6.2.0 config 1


-----Upgrade config:

[root@marathon-01 ~]# head /etc/cluster/cluster.conf
        <cluster name="marathon" config_version="9">
                <clusternodes>

[root@marathon-01 ~]# cman_tool version -r

----- On all nodes cman_tool shows:

# cman_tool version
6.2.0 config 9
# head /etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster config_version="9" name="marathon">


----- Attempt to downgrade 

[root@marathon-01 ~]# head /etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster config_version="5" name="marathon">

[root@marathon-01 ~]# cman_tool version -r
cman_tool: Error loading configuration in corosync/cman

[root@marathon-01 ~]# tail /var/log/messages
Jul 29 15:12:13 marathon-01 corosync[12263]:   [CMAN  ] Can't get updated config version: New configuration version has to be newer than current running configuration#012.


----- Check cman_tool version on all nodes:
[root@marathon-01 ~]# cman_tool version
6.2.0 config 9
[root@marathon-02 ~]# cman_tool version
6.2.0 config 9
[root@marathon-03 ~]# cman_tool version
6.2.0 config 9
[root@marathon-04 ~]# cman_tool version
6.2.0 config 9
[root@marathon-05 ~]# cman_tool version
6.2.0 config 9

But, look at the cluster.conf, it got the config_version="5" file!

[root@marathon-01 ~]# grep config_version /etc/cluster/cluster.conf
<cluster config_version="5" name="marathon">
[root@marathon-02 ~]# grep config_version /etc/cluster/cluster.conf
<cluster config_version="5" name="marathon">
[root@marathon-03 ~]# grep config_version /etc/cluster/cluster.conf
<cluster config_version="5" name="marathon">
[root@marathon-04 ~]# grep config_version /etc/cluster/cluster.conf
<cluster config_version="5" name="marathon">
[root@marathon-05 ~]# grep config_version /etc/cluster/cluster.conf
<cluster config_version="5" name="marathon">
Comment 11 Fabio Massimo Di Nitto 2010-07-30 06:24:38 UTC 
(In reply to comment #10)
> Initial sanity check failed:
> 
> 5 node cluster:
> 
> ----- On all nodes, cman_tool version shows:
> 
> #  cman_tool version
> 6.2.0 config 1
> 
> 
> -----Upgrade config:
> 
> [root@marathon-01 ~]# head /etc/cluster/cluster.conf
>         <cluster name="marathon" config_version="9">
>                 <clusternodes>
> 
> [root@marathon-01 ~]# cman_tool version -r
> 
> ----- On all nodes cman_tool shows:
> 
> # cman_tool version
> 6.2.0 config 9
> # head /etc/cluster/cluster.conf
> <?xml version="1.0"?>
> <cluster config_version="9" name="marathon">
> 
> 
> ----- Attempt to downgrade 
> 
> [root@marathon-01 ~]# head /etc/cluster/cluster.conf
> <?xml version="1.0"?>
> <cluster config_version="5" name="marathon">
> 
> [root@marathon-01 ~]# cman_tool version -r
> cman_tool: Error loading configuration in corosync/cman
> 
> [root@marathon-01 ~]# tail /var/log/messages
> Jul 29 15:12:13 marathon-01 corosync[12263]:   [CMAN  ] Can't get updated
> config version: New configuration version has to be newer than current running
> configuration#012.
> 
> 
> ----- Check cman_tool version on all nodes:
> [root@marathon-01 ~]# cman_tool version
> 6.2.0 config 9
> [root@marathon-02 ~]# cman_tool version
> 6.2.0 config 9
> [root@marathon-03 ~]# cman_tool version
> 6.2.0 config 9
> [root@marathon-04 ~]# cman_tool version
> 6.2.0 config 9
> [root@marathon-05 ~]# cman_tool version
> 6.2.0 config 9
> 
> But, look at the cluster.conf, it got the config_version="5" file!
> 
> [root@marathon-01 ~]# grep config_version /etc/cluster/cluster.conf
> <cluster config_version="5" name="marathon">
> [root@marathon-02 ~]# grep config_version /etc/cluster/cluster.conf
> <cluster config_version="5" name="marathon">
> [root@marathon-03 ~]# grep config_version /etc/cluster/cluster.conf
> <cluster config_version="5" name="marathon">
> [root@marathon-04 ~]# grep config_version /etc/cluster/cluster.conf
> <cluster config_version="5" name="marathon">
> [root@marathon-05 ~]# grep config_version /etc/cluster/cluster.conf
> <cluster config_version="5" name="marathon">    

This is a different bug.

You have not been able to downgrade the configuration in the running cluster and that´s what this bug was all about.

The issue you are seeing here is that ccs_sync will still propagate the configuration regardless. We need to clone this bug in ricci (for ccs_sync) to implement a version check prior executing propagation.

This bug is indeed fixed since the runtime version of the cluster configuration has not downgraded.
Comment 14 releng-rhel@redhat.com 2010-11-10 20:00:01 UTC 
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.
Note You need to log in before you can comment on or make changes to this bug.