Bug 2159454

Summary:	Do not call agents 'validate-all' action in 'pcs (resource\|stonith) (create\|update)' commands by default
Product:	Red Hat Enterprise Linux 9	Reporter:	Tomas Jelinek <tojeline>
Component:	pcs	Assignee:	Ondrej Mular <omular>
Status:	CLOSED ERRATA	QA Contact:	cluster-qe <cluster-qe>
Severity:	urgent	Docs Contact:	Steven J. Levine <slevine>
Priority:	urgent
Version:	9.2	CC:	cluster-maint, idevat, mlisik, mmazoure, mpospisi, nhostako, omular, pasik, slevine, tojeline
Target Milestone:	rc	Keywords:	Triaged
Target Release:	9.2
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	pcs-0.11.4-4.el9	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	2159455 (view as bug list)		Environment:
Last Closed:	2023-05-09 07:18:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2112270
Bug Blocks:	2159455

Description Tomas Jelinek 2023-01-09 16:15:21 UTC

Description of problem:
In bz2112270, calling agents 'validate-all' action from pcs was implemented. The action is called by default when creating or updating a resource. Majority of resource and stonith agents are not ready for their validate-all action to be run in this context, however. Agents provide confusing messages or report errors when they shouldn't.
In order to give agent developers time to fix these issues, pcs will call validate-all action only when requested by users and not by default.


Version-Release number of selected component (if applicable):
pcs-0.11.3-5.el9


How reproducible:
always, easily


Steps to Reproduce:
Run 'pcs (resource|stonith) (create|update)' with --debug and observe pcs calling 'crm_resource --validate'


Actual results:
pcs calls 'crm_resource --validate'


Expected results:
pcs doesn't call 'crm_resource --validate' unless explicitly requested

Comment 3 Ondrej Mular 2023-01-13 07:53:31 UTC

Upstream patch: https://github.com/ClusterLabs/pcs/commit/ef8ee600651a85dbe777f75285ffc1bbd9e021c9

Test:
[root@rhel92-node1 pcs]# pcs resource create test_resource apache --agent-validation
Assumed agent name 'ocf:heartbeat:apache' (deduced from 'apache')
Error: Validation result from agent (use --force to override):
  ocf-exit-reason:apache httpd program not found
  ocf-exit-reason:environment is invalid, resource considered stopped
Error: Errors have occurred, therefore pcs is unable to continue
[root@rhel92-node1 pcs]# echo $?
1
[root@rhel92-node1 pcs]# pcs resource create test_resource apache
Assumed agent name 'ocf:heartbeat:apache' (deduced from 'apache')
[root@rhel92-node1 pcs]# echo $?
0

Comment 4 Michal Pospisil 2023-01-13 19:32:55 UTC

DevTestResults:

[root@r09-02-c ~]# pcs resource create test_resource apache --agent-validation
Assumed agent name 'ocf:heartbeat:apache' (deduced from 'apache')
Error: Validation result from agent (use --force to override):
 ocf-exit-reason:apache httpd program not found
 ocf-exit-reason:environment is invalid, resource considered stopped
Error: Errors have occurred, therefore pcs is unable to continue

[root@r09-02-c ~]# echo $?
1

[root@r09-02-c ~]# pcs resource create test_resource apache
Assumed agent name 'ocf:heartbeat:apache' (deduced from 'apache')

[root@r09-02-c ~]# echo $?
0

Comment 8 Michal Mazourek 2023-01-30 12:07:59 UTC

BEFORE:
=======

[root@virt-152 ~]# rpm -q pcs
pcs-0.11.4-2.el9.x86_64


## Checking that the validate-all action is called by default

[root@virt-152 ~]# pcs resource create dummy ocf:heartbeat:Dummy --debug 2>&1 | grep "\-\-validate"
Running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat
Finished running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat
<pacemaker-result api-version="2.25" request="/usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat">
[root@virt-152 ~]# echo $?
0

> Validation is present when creating a resource


AFTER:
======

[root@virt-148 ~]# rpm -q pcs
pcs-0.11.4-4.el9.x86_64


### Checking in debug mode that pcs is calling the validation only if option --agent-validation is present

## Checking that the validate-all action is not called by default when creating a resource

[root@virt-148 ~]# pcs resource create dummy ocf:heartbeat:Dummy --debug 2>&1 | grep "\-\-validate"
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs resource status dummy
  * dummy	(ocf:heartbeat:Dummy):	 Started virt-148

> OK


## Checking that the validate-all action is not called by default when updating a resource

[root@virt-148 ~]# pcs resource update dummy fake=1 --debug 2>&1 | grep "\-\-validate"
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs resource config dummy | grep fake
    fake=1

> OK


## Checking that the validate-all action is not called by default when creating a stonith device

[root@virt-148 ~]# pcs stonith create sbd_fencing fence_sbd devices=invalid --debug 2>&1 | grep "\-\-validate"
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs stonith status sbd_fencing
  * sbd_fencing	(stonith:fence_sbd):	 Stopped

> OK


## Checking that the validate-all action is not called by default when updating a stonith device

[root@virt-148 ~]# pcs stonith update sbd_fencing fence_sbd devices=updated --debug 2>&1 | grep "\-\-validate"
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs stonith config sbd_fencing | grep devices
    devices=updated

> OK: pcs doesn't call the agent validation by default


[root@virt-148 ~]# pcs resource delete dummy
Attempting to stop: dummy... Stopped
[root@virt-148 ~]# pcs stonith delete sbd_fencing
Deleting Resource - sbd_fencing


## Checking that the validate-all action is called with option --agent-validation when creating a resource

[root@virt-148 ~]# pcs resource create dummy ocf:heartbeat:Dummy --agent-validation --debug 2>&1 | grep "\-\-validate"
Running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat
Finished running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat
<pacemaker-result api-version="2.25" request="/usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat">
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource status dummy
  * dummy	(ocf:heartbeat:Dummy):	 Started virt-148

> OK


## Checking that the validate-all action is called with --agent-validation when updating a resource

[root@virt-148 ~]# pcs resource update dummy fake=1 --agent-validation --debug 2>&1 | grep "\-\-validate"Running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat
Finished running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat
<pacemaker-result api-version="2.25" request="/usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat">
Running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat --option fake=1
Finished running: /usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat --option fake=1
<pacemaker-result api-version="2.25" request="/usr/sbin/crm_resource --validate --output-as xml --class ocf --agent Dummy --provider heartbeat --option fake=1">
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource config dummy | grep fake
    fake=1

> OK


## Checking that the validate-all action is called with --agent-validation when creating a stonith device

[root@virt-148 ~]# pcs stonith create sbd_fencing fence_sbd devices=invalid --agent-validation --debug 2>&1 | grep "\-\-validate"
Running: /usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=invalid
Finished running: /usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=invalid
<pacemaker-result api-version="2.25" request="/usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=invalid">
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs stonith status sbd_fencing
  * sbd_fencing	(stonith:fence_sbd):	 Stopped

> OK


## Checking that the validate-all action is called with --agent-validation when updating a stonith device

[root@virt-148 ~]# pcs stonith update sbd_fencing fence_sbd devices=updated --agent-validation --debug 2>&1 | grep "\-\-validate"
Running: /usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=invalid
Finished running: /usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=invalid
<pacemaker-result api-version="2.25" request="/usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=invalid">
Running: /usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=updated
Finished running: /usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=updated
<pacemaker-result api-version="2.25" request="/usr/sbin/stonith_admin --validate --output-as xml --agent fence_sbd --option devices=updated">
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs stonith config sbd_fencing | grep devices
    devices=updated

> OK


### Checking cases with invalid option (first with --agent-validation, then without it)

## Creating resource with invalid option with --agent-validation

[root@virt-148 ~]# pcs resource create test_ip ocf:heartbeat:IPaddr2 ip=1.1.1.1 --agent-validation
Error: Validation result from agent (use --force to override):
  Jan 24 11:12:01 ERROR: Unable to find nic or netmask.
  ocf-exit-reason:[findif] failed
Error: Errors have occurred, therefore pcs is unable to continue
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs resource status test_ip
Error: resource or tag id 'test_ip' not found
[root@virt-148 ~]# echo $?
1

> OK


## Creating resource with invalid option without --agent-validation

[root@virt-148 ~]# pcs resource create test_ip ocf:heartbeat:IPaddr2 ip=1.1.1.1
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource status test_ip
  * test_ip	(ocf:heartbeat:IPaddr2):	 Stopped

> OK: The agent validation was not called by default (without --agent-validation option)

[root@virt-148 ~]# pcs resource delete test_ip
Deleting Resource - test_ip

# Overriding the validation --agent-validation with --force option
[root@virt-148 ~]# pcs resource create test_ip ocf:heartbeat:IPaddr2 ip=1.1.1.1 --agent-validation --force
Warning: Validation result from agent:
  Jan 24 11:22:39 ERROR: Unable to find nic or netmask.
  ocf-exit-reason:[findif] failed
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource status test_ip
  * test_ip	(ocf:heartbeat:IPaddr2):	 Stopped

> OK


## Updating resource with invalid option with --agent-validation

[root@virt-148 ~]# pcs resource create test_ip_1 ocf:heartbeat:IPaddr2 ip=192.168.2.28
[root@virt-148 ~]# pcs resource status test_ip_1
  * test_ip_1	(ocf:heartbeat:IPaddr2):	 Started virt-148
[root@virt-148 ~]# pcs resource update test_ip_1 ip=invalid --agent-validation
Error: Validation result from agent (use --force to override):
  Jan 24 13:13:39 ERROR: IP address [invalid] not valid.
  ocf-exit-reason:[findif] failed
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs resource update test_ip_1 ip=1.2.3.4 --agent-validation
Error: Validation result from agent (use --force to override):
  Jan 24 13:14:04 ERROR: Unable to find nic or netmask.
  ocf-exit-reason:[findif] failed
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs resource delete test_ip_1
Attempting to stop: test_ip_1... Stopped

> OK


## Updating resource with invalid option without --agent-validation

[root@virt-148 ~]# pcs resource create test_ip_1 ocf:heartbeat:IPaddr2 ip=192.168.2.17
[root@virt-148 ~]# pcs resource status test_ip_1
  * test_ip_1	(ocf:heartbeat:IPaddr2):	 Started virt-150
[root@virt-148 ~]# pcs resource update test_ip_1 ip=invalid
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource config test_ip_1 | grep ip=
    ip=invalid

> OK


## Creating stonith device with invalid option with --agent-validation

[root@virt-148 ~]# pcs stonith create test_stonith fence_xvm ip_family=test --agent-validation
Error: Validation result from agent (use --force to override):

Error: Errors have occurred, therefore pcs is unable to continue
[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs stonith config test_stonith
Warning: Unable to find stonith device 'test_stonith'
Error: No stonith device found
[root@virt-148 ~]# echo $?
1

> OK


## Creating stonith device with invalid option without --agent-validation

[root@virt-148 ~]# pcs stonith create test_stonith fence_xvm ip_family=test
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs stonith config test_stonith | grep ip_family
    ip_family=test

> OK

# with valid option
[root@virt-148 ~]# pcs stonith create test_stonith fence_xvm ip_family=ipv4
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs stonith status test_stonith 
  * test_stonith	(stonith:fence_xvm):	 Started virt-148

> OK


## Updating stonith device with invalid option with --agent-validation

[root@virt-148 ~]# pcs stonith status test_stonith
  * test_stonith	(stonith:fence_xvm):	 Started virt-148
[root@virt-148 ~]# pcs stonith update test_stonith ip_family=test --agent-validation
Error: Validation result from agent (use --force to override):

[root@virt-148 ~]# echo $?
1

> OK


## Updating stonith device with invalid option without --agent-validation

[root@virt-148 ~]# pcs stonith update test_stonith ip_family=test
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs stonith config test_stonith | grep ip_family
    ip_family=test

> OK

[root@virt-148 ~]# pcs resource delete test_ip_1
Deleting Resource - test_ip_1


## Trying to create resource with invalid option in combination with group, promotable, disable with --agent-validation

[root@virt-148 ~]# pcs resource create test_ip_1 ocf:heartbeat:IPaddr2 ip=invalid --group g1 --agent-validation
Error: Validation result from agent (use --force to override):
  Jan 24 14:40:03 ERROR: IP address [invalid] not valid.
  ocf-exit-reason:[findif] failed
Error: Errors have occurred, therefore pcs is unable to continue
[root@virt-148 ~]# echo $?
1

[root@virt-148 ~]# pcs resource create test_ip_1 ocf:heartbeat:IPaddr2 ip=invalid promotable --agent-validation
Error: Validation result from agent (use --force to override):
  Jan 24 14:40:46 ERROR: IP address [invalid] not valid.
  ocf-exit-reason:[findif] failed
Error: Errors have occurred, therefore pcs is unable to continue
[root@virt-148 ~]# echo $?
1

[root@virt-148 ~]# pcs resource create test_ip_1 ocf:heartbeat:IPaddr2 ip=invalid --disable --agent-validation
Error: Validation result from agent (use --force to override):
  Jan 24 14:41:12 ERROR: IP address [invalid] not valid.
  ocf-exit-reason:[findif] failed
Error: Errors have occurred, therefore pcs is unable to continue
[root@virt-148 ~]# echo $?
1


## Trying to create resource with invalid option in combination with group, promotable, disable without --agent-validation

[root@virt-148 ~]# pcs resource create test_ip_1 ocf:heartbeat:IPaddr2 ip=invalid --group g1
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource create test_ip_2 ocf:heartbeat:IPaddr2 ip=1.2.3.4 promotable
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource create test_ip_3 ocf:heartbeat:IPaddr2 ip=2.3.4.5 --disable 
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource
  * test_ip	(ocf:heartbeat:IPaddr2):	 Stopped
  * Resource Group: g1:
    * test_ip_1	(ocf:heartbeat:IPaddr2):	 Stopped
  * Clone Set: test_ip_2-clone [test_ip_2] (promotable):
    * Stopped: [ virt-148 virt-150 ]
  * test_ip_3	(ocf:heartbeat:IPaddr2):	 Stopped (disabled)

> OK

[root@virt-148 ~]# pcs resource delete test_ip
Deleting Resource - test_ip
[root@virt-148 ~]# pcs resource delete test_ip_1
Deleting Resource (and group) - test_ip_1
[root@virt-148 ~]# pcs resource delete test_ip_2
Deleting Resource - test_ip_2
[root@virt-148 ~]# pcs resource delete test_ip_3
Deleting Resource - test_ip_3


## Updating disabled stonith device with invalid option with --agent-validation

[root@virt-148 ~]# pcs stonith create test_disabled fence_xvm ip_family=ipv4
[root@virt-148 ~]# pcs stonith disable test_disabled
[root@virt-148 ~]# pcs stonith status test_disabled
  * test_disabled	(stonith:fence_xvm):	 Stopped (disabled)
[root@virt-148 ~]# pcs stonith update test_disabled ip_family=invalid --agent-validation
Error: Validation result from agent (use --force to override):

[root@virt-148 ~]# echo $?
1
[root@virt-148 ~]# pcs stonith config test_disabled | grep ip_family
    ip_family=ipv4

> OK


## Updating disabled stonith device with invalid option without --agent-validation

[root@virt-148 ~]# pcs stonith update test_disabled ip_family=invalid
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs stonith config test_disabled | grep ip_family
    ip_family=invalid

> OK


## Forcing resource to have an invalid option and then updating it with another invalid option with --agent-validation

[root@virt-148 ~]# pcs resource create test_ip_invalid ocf:heartbeat:IPaddr2 ip=invalid --agent-validation --force
Warning: Validation result from agent:
  Jan 30 12:44:47 ERROR: IP address [invalid] not valid.
  ocf-exit-reason:[findif] failed
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource config test_ip_invalid | grep ip=
    ip=invalid

[root@virt-148 ~]# pcs resource update test_ip_invalid ip=1.2.3.4 --agent-validation
Warning: The resource was misconfigured before the update, therefore agent self-validation will not be run for the updated configuration. Validation output of the original configuration:
  Jan 30 12:45:45 ERROR: IP address [invalid] not valid.
  ocf-exit-reason:[findif] failed
[root@virt-148 ~]# echo $?
0
[root@virt-148 ~]# pcs resource config test_ip_invalid | grep ip=
    ip=1.2.3.4

> OK: The issue that was described in bz2151511 is fixed


## Updating resource with invalid option with another invalid option without --agent-validation

[root@virt-148 ~]# pcs resource config test_ip_invalid | grep ip=
    ip=1.2.3.4
[root@virt-148 ~]# pcs resource update test_ip_invalid ip=1.3.5.7 
[root@virt-148 ~]# pcs resource config test_ip_invalid | grep ip=
    ip=1.3.5.7

> OK


Marking as VERIFIED in pcs-0.11.4-4.el9.

Comment 10 errata-xmlrpc 2023-05-09 07:18:58 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pcs bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2151