Bug 1833173

Summary:	Pacemaker ACLs are not applied properly when scope is specified in a command
Product:	Red Hat Enterprise Linux 8	Reporter:	Reid Wahl <nwahl>
Component:	pacemaker	Assignee:	Ken Gaillot <kgaillot>
Status:	CLOSED ERRATA	QA Contact:	cluster-qe <cluster-qe>
Severity:	low	Docs Contact:
Priority:	low
Version:	8.1	CC:	cluster-maint, cluster-qe, kgaillot, msmazova
Target Milestone:	rc	Flags:	pm-rhel: mirror+
Target Release:	8.4
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	pacemaker-2.0.5-1.el8	Doc Type:	Bug Fix
Doc Text:	Cause: Pacemaker explicitly applied a deny ACL to all XML sections without a user-configured ACL, as well as implicitly denying access to any section without any ACL. When applying ACLs, Pacemaker would stop at the first deny policy found at or above the XML level of the desired change. Consequence: When a scoped command (limiting the effect to a particular CIB section) was used, the command could be wrongly denied because the explicitly added default-deny policy would incorrectly override a read or write policy applied at a higher level. Fix: Pacemaker no longer explicitly adds the default-deny policy to all XML sections, instead relying on the implicit denial when no ACL is present. Result: Scoped commands correctly respect ACLs.	Story Points:	---
Clone Of:	1833167	Environment:
Last Closed:	2021-05-18 15:26:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1885645
Bug Blocks:

Description Reid Wahl 2020-05-07 23:59:46 UTC

+++ This bug was initially created as a clone of Bug #1833167 +++

Description of problem:

A user with full write access to the CIB via ACLs is unable to create new constraints.

[testuser@fastvm-rhel-7-6-21 ~]$ pcs acl
ACLs are enabled

User: testuser
  Roles: ATSE_Admin
Role: ATSE_Admin
  Description: admin
  Permission: read xpath /cib (ATSE_Admin-read)
  Permission: write xpath /cib (ATSE_Admin-write)

[testuser@fastvm-rhel-7-6-21 ~]$ groups
testuser haclient

[testuser@fastvm-rhel-7-6-21 ~]$ pcs constraint location my_dummy prefers fastvm-rhel-7-6-21 
Error: Unable to update cib
Call cib_replace failed (-13): Permission denied

-----

Version-Release number of selected component (if applicable):

pacemaker-1.1.21-4.el7.x86_64
pacemaker-2.0.2-3.el8_1.2.x86_64

-----

How reproducible:

Always

-----

Steps to Reproduce:
1. Create an ACL role with `read xpath /cib` and `write xpath /cib` permissions.
2. Add a test user to that role.
3. Create a constraint as the test user.

-----

Actual results:

Error: Unable to update cib
Call cib_replace failed (-13): Permission denied

-----

Expected results:

Constraint created.

-----

Additional info:

I grant that this may be a procedural mistake.

Comment 1 Ken Gaillot 2020-05-16 01:10:52 UTC

This does appear to be a pacemaker bug, but due partly to misconfiguration.

Write permission includes read permission, so granting read permission (which is actually read-only) separately is not necessary. Granting them both shouldn't be a problem, because write permission should always supersede read-only permission. However it appears due to a bug, in the narrow case of a CIB replace operation for a subsection (which is what pcs does) that this is not the case. It is not specific to constraints. 

The workaround is to delete the read permission, and all operations should work as expected.

Comment 2 Reid Wahl 2020-05-29 00:13:20 UTC

(In reply to Ken Gaillot from comment #1)
> The workaround is to delete the read permission, and all operations should
> work as expected.

Am I doing something wrong? That doesn't seem to work either.

[testuser@fastvm-rhel-8-0-23 ~]$ date && pcs constraint location dummy1 prefers fastvm-rhel-8-0-23
Thu May 28 17:12:32 PDT 2020
Error: Unable to update cib
Call cib_replace failed (-13): Permission denied

[testuser@fastvm-rhel-8-0-23 ~]$ pcs acl
ACLs are enabled

User: testuser
  Roles: ATSE_Admin
Role: ATSE_Admin
  Description: admin
  Permission: write xpath /cib (ATSE_Admin-write)

Comment 3 Reid Wahl 2020-05-29 00:54:42 UTC

May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__unpack_acl:316)     debug: Unpacking ACLs for testuser
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (__xml_acl_parse_entry:141)    trace: Processing role 0x55edea25f3b0
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (__xml_acl_parse_entry:159)    debug: Unpacking referenced role: ATSE_Admin
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (__xml_acl_parse_entry:141)    trace: Processing write 0x55edea25ec90
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (__xml_acl_create:78)  trace: Using xpath: /cib
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__apply_acl:247)  trace: Applying 400 to /cib for /cib
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__apply_acl:266)  trace: Now enforcing ACL: /cib (1 matches)
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (replace_xml_child:4146)   trace: Delete match found...   <configuration>
...
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (replace_xml_child:4146)   trace: Delete match found...   </configuration>
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (xml_accept_changes:1009)  trace: Accepting changes to 0x55edea26f5b0
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__apply_acl:247)  trace: Applying 400 to /cib for /cib
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__apply_acl:266)  trace: Now enforcing ACL: /cib (1 matches)
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__apply_acl:277)  info: Enforcing default ACL for testuser to configuration
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (find_entity_by_attr_or_just_name:1807)    trace: node <rsc_location id=location-dummy1-fastvm-rhel-8-0-23-INFINITY> not found in constraints
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__check_acl:604)  trace: 400 access denied to /cib/configuration/constraints/rsc_location[@id='location-dummy1-fastvm-rhel-8-0-23-INFINITY']: parent
May 28 17:51:01 fastvm-rhel-8-0-23 pacemaker-based     [29675] (pcmk__post_process_acl:501)   trace: Cannot add new node rsc_location at /cib/configuration/constraints/rsc_location[@id='location-dummy1-fastvm-rhel-8-0-23-INFINITY']



void
pcmk__apply_acl(xmlNode *xml)
{
...
    p = xml->_private;
    if (is_not_set(p->flags, xpf_acl_read)
        && is_not_set(p->flags, xpf_acl_write)) {

        p->flags |= xpf_acl_deny;
        p = xml->doc->_private;
        crm_info("Enforcing default ACL for %s to %s",
                 p->user, crm_element_name(xml));
    }

}

Comment 4 Reid Wahl 2020-05-29 00:58:30 UTC

Adding write permission for /cib/configuration serves as a workaround:

[root@fastvm-rhel-8-0-23 ~]# pcs acl permission add ATSE_Admin write xpath /cib/configuration

[testuser@fastvm-rhel-8-0-23 ~]$ date && pcs constraint location dummy1 prefers fastvm-rhel-8-0-23
Thu May 28 17:55:32 PDT 2020
[testuser@fastvm-rhel-8-0-23 ~]$


Using only "write xpath /cib" doesn't work. Based on the output above and on the trace, I guess we're starting with the <configuration> node and ignoring the <cib> that it's nested inside? Not familiar at all with the XML libs.

Comment 5 Reid Wahl 2020-05-29 01:11:50 UTC

(In reply to Reid Wahl from comment #4)
> Based on the output above and on
> the trace, I guess we're starting with the <configuration> node and ignoring
> the <cib> that it's nested inside? Not familiar at all with the XML libs.

Issue with application of ACLs when scope is limited I'm assuming, since the pcs constraint commands use `/usr/sbin/cibadmin --replace -V --xml-pipe -o configuration`.

Comment 6 Reid Wahl 2020-05-29 04:10:16 UTC

(In reply to Reid Wahl from comment #5)
> Issue with application of ACLs when scope is limited I'm assuming, since the
> pcs constraint commands use `/usr/sbin/cibadmin --replace -V --xml-pipe -o
> configuration`.

One last little demo that seems to support that it's an issue when scope is specified.


[testuser@fastvm-rhel-8-0-23 ~]$ pcs acl
ACLs are enabled

User: testuser
  Roles: ATSE_Admin
Role: ATSE_Admin
  Description: admin
  Permission: write xpath /cib (ATSE_Admin-write)

# # Make backup of current CIB, capturing only the <configuration> section.
[testuser@fastvm-rhel-8-0-23 ~]$ pcs cluster cib scope=configuration > /tmp/cib_old.xml

# # Save a copy of the entire CIB, for modification.
[testuser@fastvm-rhel-8-0-23 ~]$ pcs cluster cib > /tmp/cib_mod.xml

# # Add a constraint to the copy of the CIB.
[testuser@fastvm-rhel-8-0-23 ~]$ pcs -f /tmp/cib_mod.xml constraint location dummy1 prefers fastvm-rhel-8-0-23

# # Push the updated copy of the whole CIB.
[testuser@fastvm-rhel-8-0-23 ~]$ pcs cluster cib-push /tmp/cib_mod.xml
CIB updated

# # Attempt to push the backup CIB <configuration> section. This fails.
[testuser@fastvm-rhel-8-0-23 ~]$ pcs cluster cib-push /tmp/cib_old.xml scope=configuration 
Error: unable to push cib
Call cib_replace failed (-13): Permission denied

Comment 7 Ken Gaillot 2020-06-18 19:49:45 UTC

You're correct. I was working with cibadmin, and must have tested without --scope.

Comment 8 Ken Gaillot 2020-08-03 14:50:37 UTC

Thanks to Reid Wahl for the fix, merged upstream as of commit ccd30cf

Comment 12 Markéta Smazová 2020-11-06 10:46:15 UTC

before fix
-----------

[testuser@virt-142 ~]$ rpm -q pacemaker
pacemaker-2.0.4-6.el8.x86_64

[testuser@virt-142 ~]$ pcs acl
ACLs are enabled

User: testuser
  Roles: TEST_Admin
Role: TEST_Admin
  Permission: write xpath /cib (TEST_Admin-write)
  
[testuser@virt-142 ~]$ groups
testuser haclient

[testuser@virt-142 ~]$ pcs status
Cluster name: STSRHTS24375
Cluster Summary:
  * Stack: corosync
  * Current DC: virt-153 (version 2.0.4-6.el8-2deceaa3ae) - partition with quorum
  * Last updated: Thu Nov  5 14:45:52 2020
  * Last change:  Thu Nov  5 14:44:10 2020 by testuser via cibadmin on virt-142
  * 2 nodes configured
  * 3 resource instances configured

Node List:
  * Online: [ virt-142 virt-153 ]

Full List of Resources:
  * fence-virt-142	(stonith:fence_xvm):	 Started virt-142
  * fence-virt-153	(stonith:fence_xvm):	 Started virt-153
  * dummy	(ocf::pacemaker:Dummy):	 Started virt-142

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

[testuser@virt-142 ~]$ pcs constraint location dummy prefers virt-153
Error: Unable to update cib
Call cib_replace failed (-13): Permission denied

[testuser@virt-142 ~]$ pcs resource status
  * dummy	(ocf::pacemaker:Dummy):	 Started virt-142



after fix
----------
[testuser@virt-245 ~]$ rpm -q pacemaker
pacemaker-2.0.5-2.el8.x86_64

[testuser@virt-245 ~]$ pcs acl
ACLs are enabled

User: testuser
  Roles: TEST_Admin
Role: TEST_Admin
  Permission: write xpath /cib (TEST_Admin-write)

[testuser@virt-245 ~]$ groups
testuser haclient

[testuser@virt-245 ~]$ pcs status
Cluster name: STSRHTS30795
Cluster Summary:
  * Stack: corosync
  * Current DC: virt-246 (version 2.0.5-2.el8-31aa4f5515) - partition with quorum
  * Last updated: Thu Nov  5 18:04:54 2020
  * Last change:  Thu Nov  5 18:02:13 2020 by root via cibadmin on virt-245
  * 2 nodes configured
  * 3 resource instances configured

Node List:
  * Online: [ virt-245 virt-246 ]

Full List of Resources:
  * fence-virt-245	(stonith:fence_xvm):	 Started virt-245
  * fence-virt-246	(stonith:fence_xvm):	 Started virt-246
  * dummy	(ocf::pacemaker:Dummy):	 Started virt-245

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

[testuser@virt-245 ~]$ pcs constraint location dummy prefers virt-246
[testuser@virt-245 ~]$ echo $?
0

[testuser@virt-245 ~]$ pcs resource status
  * dummy	(ocf::pacemaker:Dummy):	 Started virt-246
  
marking verified in pacemaker-2.0.5-2.el8

Comment 14 errata-xmlrpc 2021-05-18 15:26:41 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pacemaker bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:1782