Bug 1115608

Summary: Colocating Sets of Resources: 'setoptions' is invalid
Product: Red Hat Enterprise Linux 7 Reporter: tengel
Component: pcsAssignee: Tomas Jelinek <tojeline>
Status: CLOSED ERRATA QA Contact: michal novacek <mnovacek>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.0CC: cfeist, cluster-maint, dvossel, mnovacek, rsteiger, slevine, tojeline
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: pcs-0.9.126-1.el7 Doc Type: Bug Fix
Doc Text:
Cause: User enters invalid parameters in 'pcs constraint colocation set' command. Consequence: pcs creates invalid CIB and tries to push it to the cluster. That results in invalid CIB dumped into the terminal. Fix: Clarify documentation. Add validation of 'pcs constraint colocation set' options. Result: User is able to enter the command correctly based on the documentation. pcs reports errors in options.
 Story Points: ---
Clone Of:
: 1129860 (view as bug list) Environment:
Last Closed: 2015-03-05 09:20:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1129860    
Attachments:
Description Flags
proposed fix none

Description tengel 2014-07-02 18:27:51 UTC 
Description of problem:

The use of "setoptions" results an an invalid schema error; it appears this is an upstream problem. Per the RH docs section 6.3.3:

 https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/High_Availability_Add-On_Reference/s1-colocationconstraints-HAAR.html

 "colocation set resource1 resource2 [resourceN]... [setoptions name=value] ... [set resourceX resourceY ...] [setoptions name=value...]"

Attempting to use it that way results in an error:

 # pcs constraint colocation set \
    db_ip db_lvm db_fs db_srv \
    setoptions sequential=true

 Error: Unable to update cib
 Call cib_replace failed (-203): Update does not conform to the configured schema

However upon searching I found this:

 http://oss.clusterlabs.org/pipermail/pacemaker/2014-February/021170.html

I dropped 'setoptions' from the CLI and it seems to work as expected:

 # pcs constraint colocation set \
    db_ip db_lvm db_fs db_srv \
    sequential=true

 # pcs constraint show --full
 Location Constraints:
 Ordering Constraints:
 Colocation Constraints:
   Resource Sets:
     set db_ip db_lvm db_fs db_srv sequential=true (id:pcs_rsc_set_db_ip_db_lvm_db_fs_db_srv-1) setoptions score=INFINITY (id:pcs_rsc_colocation_db_ip_db_lvm_db_fs_db_srv)


Version-Release number of selected component (if applicable):

# rpm -q pacemaker
pacemaker-1.1.10-31.el7_0.x86_64


How reproducible:

100% (always)


Steps to Reproduce:

# pcs resource create db_ip IPaddr2 ip=10.12.91.5 cidr_netmask=27 --group db-svc

# pcs resource create db_lvm LVM volgrpname=vgsan00 exclusive=true --group db-svc

# pcs resource create db_fs Filesystem device="/dev/vgsan00/data00" \
  directory="/data" fstype="xfs" options="inode64,nobarrier" --group db-svc

# pcs resource create db_srv mysql \
  config="/etc/my.cnf" enable_creation="0" \
  user="mysql" group="mysql" \
  datadir="/data/mysql" \
  socket="/data/mysql/mysql.sock" \
  additional_parameters="--bind-address=10.12.91.5" \
  --group db-svc \
  op monitor interval="10s" \
  op start interval="0" timeout="300s" \
  op stop interval="0" timeout="300s"

# pcs constraint colocation set \
  db_ip db_lvm db_fs db_srv \
  sequential=true


Actual results:

 Error: Unable to update cib
 Call cib_replace failed (-203): Update does not conform to the configured schema


Expected results:

(silent output, worked)

 # pcs constraint show --full
 Location Constraints:
 Ordering Constraints:
 Colocation Constraints:
   Resource Sets:
     set db_ip db_lvm db_fs db_srv sequential=true (id:pcs_rsc_set_db_ip_db_lvm_db_fs_db_srv-1) setoptions score=INFINITY (id:pcs_rsc_colocation_db_ip_db_lvm_db_fs_db_srv)


Additional info:

Being new to PCS, I'm trying to reproduce the design of RHEL6 and earlier parent/child relationships in cluster.conf and ensure they stick together and start/stop in order ala:

service = db-svc
  resource = db_ip (IP)
    resource = db_lvm (LVM VG)
      resource = db_fs (XFS FS)
        resource = db_srv / (MariaDB)
      /resource
    /resource
  /resource
/service

I may be doing it wrong with the pcs constraint command, but I don't think that affects this bug report. The other way that the docs indicate it might work is to do this instead, I would love feedback which way is "right" in the new PCS style so we can develop best practices.

pcs constraint order set db_ip db_lvm db_fs db_srv
pcs constraint colocation add db_lvm with db_ip
pcs constraint colocation add db_fs with db_lvm
pcs constraint colocation add db_srv with db_fs
Comment 2 Tomas Jelinek 2014-07-16 12:54:47 UTC 
Created attachment 918398 [details]
proposed fix
Comment 4 Tomas Jelinek 2014-09-11 09:45:27 UTC 
Before Fix:
[root@rh70-node1 ~]# rpm -q pcs
pcs-0.9.115-32.el7.x86_64
[root@rh70-node1 ~]# man pcs
{output trimmed}
       colocation set <resource1> <resource2> [resourceN]... [setoptions] ...
              [set <resourceX> <resourceY> ...] [setoptions <name>=<value>...]
              Create a colocation constraint with a resource set
{output trimmed}
[root@rh70-node1 ~]# pcs constraint colocation set dummy1 dummy2 setoptions sequential=true
Error: Unable to update cib
Call cib_replace failed (-203): Update does not conform to the configured schema
<cib admin_epoch="0" cib-last-written="Wed Sep 10 17:23:29 2014" crm_feature_set="3.0.7" epoch="6523" have-quorum="1" num_updates="1" update-client="cibadmin" update-origin="rh70-node1" validate-with="pacemaker-1.2" dc-uuid="1">
{output trimmed}

After Fix:
[root@rh70-node1 ~]# rpm -q pcs
pcs-0.9.126-1.el7.x86_64
[root@rh70-node1 ~]# man pcs
{output trimmed}
       colocation  set  <resource1>  <resource2> [resourceN]... [options] [set
       <resourceX>    <resourceY>    ...    [options]]    [setoptions    [con‐
       straint_options]]
              Create  a  colocation  constraint with a resource set. Available
              options   are   sequential=true/false,   require-all=true/false,
              action=start/promote/demote/stop  and  role=Stopped/Started/Mas‐
              ter/Slave.  Available  constraint_options  are   score,   score-
              attribute and score-attribute-mangle.
{output trimmed}
[root@rh70-node1 ~]# pcs constraint colocation set dummy1 dummy2 setoptions sequential=true
Error: invalid option 'sequential', allowed options are: score, score-attribute, score-attribute-mangle
[root@rh70-node1 ~]# echo $?
1
[root@rh70-node1 ~]# pcs constraint colocation set dummy1 dummy2 sequential=true
[root@rh70-node1 ~]# echo $?
0
[root@rh70-node1 ~]# pcs constraint
Location Constraints:
Ordering Constraints:
Colocation Constraints:
  Resource Sets:
    set dummy1 dummy2 sequential=true setoptions score=INFINITY
Comment 6 michal novacek 2015-01-16 15:02:34 UTC 
I have verified that setoptions parameter can be used according to the manual with pcs-0.9.137-11.el7.x86_64.

-----

[root@duck-01 ~]# rpm -q pcs
pcs-0.9.137-11.el7.x86_64

[root@duck-01 ~]# pcs constraint colocation -h

Usage: pcs constraint [constraints]...
    colocation show [--full]
        List all current colocation constraints (if '--full' is specified show
        the internal constraint id's as well).

    colocation add [master|slave] <source resource id> with [master|slave]
                   <target resource id> [score] [options] [id=constraint-id]
        Request <source resource> to run on the same node where pacemaker has
        determined <target resource> should run.  Positive values of score
        mean the resources should be run on the same node, negative values
        mean the resources should not be run on the same node.  Specifying
        'INFINITY' (or '-INFINITY') for the score force <source resource> to
        run (or not run) with <target resource>. (score defaults to "INFINITY")
        A role can be master or slave (if no role is specified, it defaults to
        'started').

    colocation set <resource1> <resource2> [resourceN]... [options]
               [set <resourceX> <resourceY> ... [options]]
               [setoptions [constraint_options]]
        Create a colocation constraint with a resource set.
        Available options are sequential=true/false, require-all=true/false,
        action=start/promote/demote/stop and role=Stopped/Started/Master/Slave.
        Available constraint_options are id, score, score-attribute and
        score-attribute-mangle.

    colocation remove <source resource id> <target resource id>
        Remove colocation constraints with <source resource>

[root@duck-01 ~]# pcs resource create dummy1 Dummy
[root@duck-01 ~]# pcs resource create dummy2 Dummy
[root@duck-01 ~]# pcs resource
 Clone Set: dlm-clone [dlm]
     Started: [ duck-01 duck-02 duck-03 ]
 Clone Set: clvmd-clone [clvmd]
     Started: [ duck-01 duck-02 duck-03 ]
 dummy1 (ocf::heartbeat:Dummy): Started 
 dummy2 (ocf::heartbeat:Dummy): Started 
[root@duck-01 ~]# pcs constraint colocation set dummy1 dummy2 setoptions sequential=true
Error: invalid option 'sequential', allowed options are: score, score-attribute, score-attribute-mangle, id
[root@duck-01 ~]# pcs constraint colocation set dummy1 dummy2 sequential=true
[root@duck-01 ~]# pcs constraint
Location Constraints:
Ordering Constraints:
  start dlm-clone then start clvmd-clone (kind:Mandatory)
Colocation Constraints:
  clvmd-clone with dlm-clone (score:INFINITY)
  Resource Sets:
    set dummy1 dummy2 sequential=true setoptions score=INFINITY
[root@duck-01 ~]#
Comment 8 errata-xmlrpc 2015-03-05 09:20:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0415.html